PyCon 2010 wrap-ups
I'm back from PyCon 2010 in Atlanta (actually I've been back for almost two weeks now, but I'm a lazy writer). PyCon, as usual, was a great time. I wrote a long wrap-up on my work blog here:
A couple more personal additions...
Read more » 0 commentsSmall Django patch to add full traceback for wrapper exception during template rendering
Here is a patch that I'm tending to apply to my Django (currently v1.1.1) trees to give me a full traceback on the wrapped exception when getting an exception during template processing. Without this patch you only get the string summary of the underlying exception -- often far from enough info to track down the actual bug. With this patch the result isn't that pretty (a full Python traceback as the string summary of the TemplateSyntaxError), but the info sure is helpful.
Read more » 1 commentsKomodo 5.1.3 released
We (ActiveState) released Komodo 5.1.3 today. Get it here:
Or, if you are currently running any previous Komodo 5, click "Help > Check for Updates...". This is a bug fix release and is recommended for all users. See below for details.
Read more » 6 commentsunladden swallow: a (potentially *much*) faster CPython
Discussed a bit at the Python Language Summit at PyCon this morning: unladen-swallow is a Google project to do a lot of performance work on CPython's VM.
- Currently have about 30% speed up.
- Currently for Python 2 (2.4, I think).
- Currently focused on Linux and Python 3, but committed to get patches back to the core (which implies Python 3 support). "This is a branch, not a fork."
- Currently in use on Youtube (where most of the frontend is Python).
They are shooting for a 5x speedup. From the ProjectPlan:
Our long-term proposal is to replace CPython's custom virtual machine with a JIT built on top of LLVM, while leaving the rest of the Python runtime relatively intact. We have observed that Python applications spend a large portion of their time in the main eval loop. In particular, even relatively minor changes to VM components such as opcode dispatch have a significant effect on Python application performance. We believe that compiling Python to machine code via LLVM's JIT engine will deliver large performance benefits.
Jesse has a good write-up.
2 commentsKomodo 5.1 released (fast open, history, hyperlinks, etc.)
We (ActiveState) released Komodo 5.1 today! Get it here:
Komodo IDE 5.1 is a free upgrade for Komodo IDE 5.x license holders. Your license entitles you to run Komodo IDE on any of the platforms we support (Windows, Mac OS X and Linux). Komodo Edit 5.1 is, as ever, open-source and free.
Read more » 26 commentsActivePython 2.6.1.1 and 3.0.0.0 released!
All ActivePython downloads are also available here.
Python 3.0 was released last week and we've 1 managed to get ActivePython builds for all our platforms and give them a smoke test.
Now that Python 3.0 is final I'm really looking forward to seeing where Python 3 goes. I'm sure it will be (and feel like) a long transition from Python 2, but I think a year or two from now we'll look back at the rare Python 2 usage not miss it. The semantic changes involved in the str/unicode/bytes changes will be a royal pain, but this had to be done and the long term benefits of sane Unicode handling will be huge.
-
For the last few years I've been maintaining ActivePython myself. For these latest releases I'm happy to have Andreas Kupries helping out with ActivePython build engineering. ↩
html5lib rocks (and a patch to preserve attribute order)
I've been playing with the Python html5lib package -- having come across it reading Sam Ruby's blog. What a fantastically useful library!
Originally my interest in it was with the discussion surrounding santization, and I expect to use it for that later, but today I've been playing with some general parse/filter/serialize code to support some preprocessing of HTML documentation for Open Komodo.
My code looks like this:
import sys
from html5lib import treebuilders, treewalkers
from html5lib.serializer.xhtmlserializer import XHTMLSerializer
def filter_play(path):
p = html5lib.XHTMLParser(tree=treebuilders.getTreeBuilder("simpletree"))
f = open(path)
dom = p.parse(f)
walker = treewalkers.getTreeWalker("simpletree")
stream = walker(dom)
#stream = MyPreprocessingFilter()
s = XHTMLSerializer()
outputter = s.serialize(stream)
for item in outputter:
sys.stdout.write(item)
filter_play(sys.argv[1])
One thing that bugged me a little with the output generated with this is that attributes on HTML elements get sorted, i.e. their order is not preserved. While totally cool for correctness, this reduces the utility of using diff or similar for comparing input with output. As well, I work on the Komodo IDE/editor and would like to consider using html5lib for an HTML reflow/beautifier feature at some point. Preserving attribute order for this will be important.
To that end, here is a small patch that adds the ability to preserve attribute order in serialized output. To use it:
- You need odict.py.
You need to change the above code to:
... s = XHTMLSerializer(preserve_attr_order=True) ...
Obviously this isn't something that would be ready to check-in to html5lib. Reasons why:
- It only works for the "simpletree" treebuilder/treewalker. I'm not sure if it is feasible/practical to get it to work with some of the others (e.g. dom).
- It unconditionally requires an external non-standard module (
odict.py). - It should be optional on the parser because (a) using
OrderedDictinstead ofdictwould presumably have an undesired perf impact and (b) the attribute order normalization could be desirable for many users.
Maybe a better solution would be a custom "roundtriptree" tree type? Anyway, just throwing this up here to perhaps come back to later. I have to dig into the html5lib discussion list to see if this has come up before.
building MSI patch packages (.msp) with WiX
This post includes a complete and concrete example of building an MSI patch package (a .msp file to upgrade an existing .msi installation) with WiX.
Background
I'm responsible for building the ActivePython and Komodo installers at ActiveState. On Windows we build MSI packages for installation.
Currently I'm investigating auto-update support for Komodo 4.2. Because Komodo is based on Firefox/Mozilla we can benefit from the excellent Mozilla update system (I'll write another post about our experience with it). However, integrating with an MSI-based installation isn't something the Mozilla update system does out of the box: Firefox and Thunderbird don't use MSI for their installers (they use NSIS), hence I suspect working with MSI was never a design consideration.
While working out how to best marry MSI and Moz update, I investigated producing MSI patch packages (.msp files) for Komodo updates. MSI is a complex and complicated technology (it would be nice if the latter, at least, wasn't the case). Back in the day I used InstallShield for building our MSI packages, but now WiX is the best way to build .msi's -- by far. WiX helps a lot, but building appropriate MSI packages is still quite difficult. The following two pages helped me get to successfully building .msp's. Hopefully this concrete example will help others too.
ActiveFoo 1.0
For this example we'll build .msi installers for versions 1.0.0 and 1.0.1 of the the mythical "ActiveFoo" app ("activefoo-1.0.0.msi" and "activefoo-1.0.1.msi"). Then we'll build a '.msp' that will upgrade a 1.0.0 install to 1.0.1. We'll have the following files:
1.0.0/
activefoo.wxs # This describes "activefoo-1.0.0.msi"
config.wxi
installimage/ # The ActiveFoo install image
CHANGES.txt
foo.exe
README.txt
1.0.1/
activefoo.wxs # This describes "activefoo-1.0.1.msi"
config.wxi
installimage/ # The install image with changes for 1.0.1
CHANGES.txt
README.txt
upgrade-1.0.0.wxs # This describes the '.msp'.
make.py # 'python make.py' to build everything
README.txt
Here is a zip of the working files for this example, if you'd like to play along.
We have a simple install image with three files (foo.exe, README.txt and CHANGES.txt). The WiX code to build an installer for ActiveFoo 1.0.0 is 1.0.0/activefoo.wxs:
<?xml version="1.0" encoding="utf-8"?>
<?include config.wxi ?>
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
<Product Name="$(var.ProductName)" Id="$(var.ProductCode)"
Language="1033" Codepage="1252" Version="$(var.ProductVersion)"
Manufacturer="Acme" UpgradeCode="$(var.UpgradeCode)">
<Package Id="????????-????-????-????-????????????" Keywords="Installer"
Description="$(var.ProductName)"
Comments="blah blah" Manufacturer="Acme"
InstallerVersion="200" Languages="1033" Compressed="yes"
SummaryCodepage="1252" />
<Media Id="1" Cabinet="media.cab" EmbedCab="yes" />
<!-- Define some of the dir-structure. -->
<Directory Id="TARGETDIR" Name="SourceDir">
<Directory Id="ProgramFilesFolder" Name="PFILES">
<Directory Id="INSTALLDIR" Name="$(var.InstallId)"
LongName="$(var.InstallName)" />
</Directory>
</Directory>
<!-- Define the feature hierarchy (just one feature in this simple
example). -->
<Property Id="INSTALLLEVEL" Value="1000" />
<Feature Id="core" Title="ActiveFoo" Description="The Foo core"
Level="1">
<ComponentRef Id="MainExe" />
<ComponentRef Id="ReadMeFiles" />
</Feature>
<!-- Define all the components. -->
<DirectoryRef Id="INSTALLDIR">
<Component Id="MainExe" Guid="6ee6fda3-6f50-47bf-99b9-6031c720428e">
<File Id="MainExe" Name="foo.exe" DiskId="1"
src="installimage\foo.exe" Vital="yes" />
</Component>
<Component Id="ReadMeFiles" DiskId="1"
Guid="8f2255f3-3eaf-4c82-9688-3545cd9b2018">
<File Id="README.txt" Name="README.txt"
src="installimage\README.txt" />
<File Id="CHANGES.txt" Name="CHANGES.txt"
src="installimage\CHANGES.txt" />
</Component>
</DirectoryRef>
</Product>
</Wix>
with some configuration variables included from 1.0.0/config.wxi:
<?xml version="1.0" encoding="utf-8"?> <Include> <?define ProductCode = "cdc5e50f-b490-4a37-8ff6-22e3cb3d690e" ?> <?define UpgradeCode = "ed340ed8-aa91-4bf6-9dcf-d7f6f4d43737" ?> <?define ProductName = "ActiveFoo" ?> <?define InstallName = "ActiveFoo" ?> <?define InstallId = "AFoo10" ?> <?define ProductVersion = "1.0.0" ?> <?define ProductURL = "http://www.example.com/products/activefoo/" ?> </Include>
(Note that this WiX project is simplistic. In a real world WiX project you'd likely have a UI element for a user UI, define Add/Remove Programs -- ARP -- properties, etc.)
Use the provided "make.py" script to build "activefoo-1.0.0.msi":
C:\tmp\wix_and_msp> python make.py -v 100 INFO:make:build target '100' DEBUG:make:running 'candle -nologo activefoo.wxs' in '1.0.0' activefoo.wxs DEBUG:make:running 'light -nologo -o ../activefoo-1.0.0.msi activefoo.wixobj' in '1.0.0' INFO:make:'activefoo-1.0.0.msi' created
and install it. You should now have a "ActiveFoo" folder in your "Program Files".
ActiveFoo 1.0.1
Version 1.0.1 has the following changes:
- The ProductVersion is incremented to 1.0.1. We aren't change the ProductCode so this qualifies in MSI parlance as a "minor upgrade", as opposed to a "small update" or a "major upgrade").
- We've added a note to "CHANGES.txt" for the new release.
- We've removed the "foo.exe" file from the install image. This is so we can see how file removal can be accomplished with a "minor upgrade". There is a lot of documentation out there than says that file removal can't be done with an MSI minor upgrade. We'll see that that isn't true. I haven't seen any justification for why minor upgrades shouldn't remove files.
Normally, for these changes, the only updates to the WiX sources to build "activefoo-1.0.1.msi" would be to (a) update the "ProductVersion" string and (b) remove the File and Component elements for "foo.exe". However, working from this comment in Minor and Major Upgrades Using IPWI:
If you need to remove any files or registry data during the upgrade, add
records to the RemoveFile or RemoveRegistry tables of the newer database.
I've found that to get WiX to put a RemoveFile entry for, in this case, "foo.exe", I needed to add an explicit RemoveFile element:
...
<Component Id="MainExe" Guid="6ee6fda3-6f50-47bf-99b9-6031c720428e">
<!-- Note: This is how to explicitly remove files in an update. -->
<RemoveFile Id="removefile1" On="install" Name="foo.exe"/>
</Component>
...
The ProductVersion we updated in "1.0.1\config.wxi":
C:\tmp\wix_and_msp>diff -u 1.0.0\config.wxi 1.0.1\config.wxi --- 1.0.0\config.wxi Mon May 28 17:33:01 2007 +++ 1.0.1\config.wxi Mon May 28 17:33:03 2007 @@ -6,7 +6,7 @@ <?define ProductName = "ActiveFoo" ?> <?define InstallName = "ActiveFoo" ?> <?define InstallId = "AFoo10" ?> - <?define ProductVersion = "1.0.0" ?> + <?define ProductVersion = "1.0.1" ?> <?define ProductURL = "http://www.example.com/products/activefoo/" ?> </Include>
Now we can build "activefoo-1.0.1.msi":
C:\tmp\wix_and_msp> python make.py -v 101 INFO:make:build target '101' DEBUG:make:running 'candle -nologo activefoo.wxs' in '1.0.1' activefoo.wxs DEBUG:make:running 'light -nologo -o ../activefoo-1.0.1.msi activefoo.wixobj' in '1.0.1' INFO:make:'activefoo-1.0.1.msi' created
ActiveFoo 1.0.1 update
The basic process for building a '.msp' is:
- Get an administrative install of the old version. I hadn't known this
before: An administrative install effective just extracts the file payload
from an .msi into a given directory leaving a lighter .msi with just the
MSI database tables. AFAIK this is the same thing as if you had built an
"uncompressed MSI" -- i.e. one in which
<Package Compressed='no' .../>. make.py will put this in "1.0.1\build\before". - Get an administrative install of the new version. make.py will put this in "1.0.1\build\after".
- Write a WiX file that describes the patch.
- Compile to a Patch Creation Properties (.pcp) file with WiX.
- Compile to a '.msp' file with the "msimsp.exe" utility from the MSI SDK (Part of the Microsoft Platform SDK).
Here is a our WiX file describing the patch (1.0.1\upgrade-1.0.0.wxs) with comments inline:
<?xml version='1.0' encoding='windows-1252'?>
<Wix xmlns='http://schemas.microsoft.com/wix/2003/01/wi'>
<!-- TODO: Update PatchCreation Id for each new patch.
Can we just use WiX's '????????-????-????-????-????????????' ? -->
<PatchCreation Id='e8ee6400-7877-47e4-9519-ce17e3f1d59b'
CleanWorkingFolder='yes'
WholeFilesOnly='no'
AllowMajorVersionMismatches='yes'
AllowProductCodeMismatches='no'>
<PatchInformation Description="ActiveFoo 1.0.1 Patch"
Comments='blah blah'
Manufacturer='Acme'
Languages='1033'
Compressed='yes' />
<!-- TODO: Play with other values of 'Classification'. Does msiexec's
behaviour actually change for different values? -->
<PatchMetadata Description="ActiveFoo 1.0.1 Patch"
DisplayName="ActiveFoo 1.0.1 Patch"
TargetProductName='ActiveFoo 1.0'
ManufacturerName='Acme'
MoreInfoURL='http://www.example.com/products/activefoo'
Classification='Update'
AllowRemoval='yes' />
<!-- From <http://wix.sourceforge.net/manual-wix2/patch_building.htm>
"""
The SequenceStart value is influenced by the number of files that
the previous patch delivered, as well as the number of files that
this patch will deliver. This tells PatchWiz.dll to start assigning
File sequence numbers from this number. So if this patch ships 11
files, and the next patch uses a SequenceStart of 1020, it will step
on the 11th file's assigned sequence number. In this case the next
patch would use a SequenceStart of 1030, and 03 as the patch id to
avoid conflicts with this patch. This scheme helps prevent this by
coordinating the SequenceStart (file sequence numbers) with the
patch sequence number. Also, note that the SequenceStart of the
first patch must be greater than the number of files in the original
installation. If the original installation contained more than 1000
files(rare), then the SequenceStart for the first patch must be set
to a higher value (e.g 2010.)
"""
-->
<!-- Name is max 8 chars. *How* unique does this have to be? -->
<Family Name='Fam101' DiskId='2' MediaSrcProp='AFoo10_2_1_01'
SequenceStart='1010'>
<UpgradeImage Id='AFoo10Upgrade'
SourceFile='after\activefoo-1.0.1.msi'>
<TargetImage Id='AFoo10Target' Order='1' IgnoreMissingFiles='no'
SourceFile='before\activefoo-1.0.0.msi' />
</UpgradeImage>
</Family>
<TargetProductCode Id='cdc5e50f-b490-4a37-8ff6-22e3cb3d690e' />
</PatchCreation>
</Wix>
Use make.py to build the patch:
C:\tmp\wix_and_msp> python make.py -v 101_upgrade
INFO:make:build target '101_upgrade'
DEBUG:make:running 'msiexec /a activefoo-1.0.0.msi TARGETDIR=C:\tmp\wix_and_msp\1.0.1\build\before'
DEBUG:make:running 'msiexec /a activefoo-1.0.1.msi TARGETDIR=C:\tmp\wix_and_msp\1.0.1\build\after'
1 file(s) copied.
DEBUG:make:running 'candle -nologo upgrade.wxs' in 'C:\tmp\wix_and_msp\1.0.1\build'
upgrade.wxs
DEBUG:make:running 'light -nologo upgrade.wixobj' in 'C:\tmp\wix_and_msp\1.0.1\build'
DEBUG:make:running '"C:\Program Files\Microsoft Platform SDK\Samples\SysMgmt\Msi\Patching\MsiMsp.Exe" -s upgrade.pcp -p C:\tmp\wix_and_msp\activefoo-1.0.1-upgrade-1.0.0.msp -l upgrade.log' in 'C:\tmp\wix_and_msp\1.0.1\build'
INFO:make:'activefoo-1.0.1-upgrade-1.0.0.msp' created
INFO:make:To install the update, run:
msiexec /p activefoo-1.0.1-upgrade-1.0.0.msp REINSTALL=ALL REINSTALLMODE=omus
You should now be able to install "activefoo-1.0.1-upgrade-1.0.0.msp" over an ActiveFoo 1.0.0 installation to upgrade to ActiveFoo 1.0.1. Note that some docs out there mention an MSI bug preventing installation of a '.msp' by double-clicking on that. I've found that I am able to install by double-clicking on my WinXP box with Windows Installer V 3.01.4000.1823.
Notes/Limitations
- Having to explicitly put in RemoveFile elements to ensure upgrades remove them is a pain. It would be nice if WiX inferred that automatically. WiX v3 is slated to include "Patch creation support" and "ClickThrough". Perhaps these will go a long way to making all of this easier.
- There are many variables to tweak here that I haven't played with. I haven't deployed any .msp's built as describe here to users on any scale so I there may be gremlins lurking in this procedure.
I'd be happy to hear about others' experiences working with WiX and MSI patches.
1 commentsKomodo Planet and Yahoo Pipes
I played a bit with Yahoo's new Pipes web app. At ActiveState we have a number of blogs where many of us write about what interesting things we are working on. A number of those posts are about Komodo. As well, with the recent 4.0 release, Komodo has started popping up on blogs more and more -- in particular, the very cool "Komodo Hacks" series from Brandon.
So, without further ado, I give you the Komodo Planet Pipe.
Let me know if there are other blogs I should add to this. Currently it just filters on posting titles matching "Komodo". I tried to also include filtering on the posting content but Yahoo Pipes kept giving me (unhelpful) errors with that. I like the idea of Pipes, but overall I found the current experience to be too unresponsive to be pleasing.
0 commentsKomodo babies
Long time, no post. I've been working on Komodo 4.0 -- which we finally released yesterday after over a year of work! Top that off with the recent announcement by the British Zoo of the virgin birth of 5 baby Komodo dragons! Perfect.
Komodo 4.0 adds JavaScript debugging (amongst a bunch of other features) and
the reptiles are in good health and enjoying a diet of crickets and locusts.0 comments
Firefox and Thunderbird featurelets I want
I use Firefox and Thunderbird. (For the latter I recently switched from using mutt -- the last luddite at work to finally switch to one of them new fangled GUI things for email.) Here are a couple fixes/featurelets that I want in them.
- I set Firefox to open new URLs in a new tab. This is the behaviour I want most of the time. However, there is the odd time that I want to lock a particular browser window. I.e., I'd like subsequent decisions on where to open a new URL to ignore this window. For example, while editing this blog posting in my browser, I would like new URLs (ones I'm hunting for to link from this post) to open in another browser window. Ditto when I'm working on my calendar in the browser. Or in gmail. What if Firefox had a "pin" button (as provided in some Linux window managers)? That would be handy.
- Say you previously sent this email:
To: friend@example.com From: me@example.com Subject: Here are some pics from my weekendAnd, of course, you forgot to attach the pics. So you reply to your own email. In mutt and gmail when you reply to that email you get:To: friend@example.com From: me@example.com Subject: Re: Here are some pics from my weekendIn Thunderbird, when you "Reply All" you get these headers:To: me@example.com Cc: friend@example.com Subject: Re: Here are some pics from my weekendThat is just wrong. Even worse is when you just hit "Reply":To: me@example.com Subject: Re: Here are some pics from my weekendI guess your friend just won't see how your weekend went. [Update: I couldn't find a bug for this so I added this one.] [Update 2: ... and quickly closed: "fixed for 2.0". Cool.]
thirtyboxes.py 0.5.0: a much nicer interface than 0.1.0
I've put up a new thirtyboxes.py binding to the 30boxes web API: http://code.google.com/p/python-thirtyboxes/
For example:
>>> import thirtyboxes
>>> tb = thirtyboxes.ThirtyBoxes()
>>> tb.find_user(1)
{'avatar': 'http://static.flickr.com/25/97988637_27ec96a24f_o.jpg',
'createDate': datetime.date(2005, 9, 10),
'firstName': 'Nick',
'id': 1,
...}
>>> from datetime import date, timedelta
>>> today = date.today()
>>> tomorrow = today + timedelta(1)
>>> tb.events(start=today, end=tomorrow)
{'events': [{'allDayEvent': False,
'end': datetime.datetime(2006, 3, 25, 22, 0),
'id': 156569,
'invitation': {'isInvitation': False},
'notes': ''
'privacy': 'shared',
'repeatEndDate': None,
'repeatType': 'no',
'start': datetime.datetime(2006, 3, 25, 19, 0),
'summary': 'Bagpipe practice',
'tags': 'pipes'}],
'listEnd': datetime.date(2006, 3, 26),
'listStart': datetime.date(2006, 3, 25),
'userId': 1234}
>>> tb.search('caber toss')
...returns events for caber tossing
Or, from the command line
$ alias 30b='python -m thirtyboxes.py' $ 30b user 1234 --- 30boxes user name : Hamish McDonald id : 1234 personalSite : http://hamish.example.com/ avatar : ... createDate : 2006-02-05 startDay : 0 use24HourClock : False feeds : - hamish's Photos (http://www.flickr.com/services/feeds/pho...
Any feedback is appreciated.
0 commentsthirtyboxes.py -- a Python 30boxes.com API
30boxes.com is a nice web calendaring service, maligned somewhat by Joel Spolsky. I've been using Trumba for a while, but have now switched to 30boxes. So far I am pretty happy with the change. The "One Box" (any name for it would have been cheesy) is a great answer to the tedium of adding events quickly.
30boxes.com has started adding a web API as promised (it is read-only right now). I've cobbled together a Python binding to this API. I hope some folks find it useful. Please let me know if you love it, hate it, have problems with it, whatever.
$ python
>>> import thirtyboxes
>>> thirtyboxes.events_TagSearch("work")
'<?xml version="1.0" encoding="ISO-8859-1"?>
<rsp stat="ok">
<eventlist>
...'
cygwin + NTFS permissions = badness
Cygwin is "a Linux-like environment for Windows." Basically it provides all (or at least a large set) of the standard Linux command line tools for Windows. A lot of open source projects (e.g. Mozilla, for a big one) have based their Windows build systems on Cygwin because it simplifies the problems with trying to get a make-based build system to work on Windows.
NTFS (the file system for WinNT, Win2k, WinXP, Win2k3) uses Access Control Lists (ACLs) for managing file permissions. If you have ever been frustrated by not being able delete a file on Windows: your NTFS ACLs might be the culprit.
I don't pretend to fully understand NTFS ACLs, but follow along with this little experiment and decide if you think there is a problem waiting to happen here. For this experiment you'll need xcacls.exe from the Windows Resource Kit. This is a little command-line tool for dumping NTFS ACL information. You can also view ACL information by opening the "Propeties" dialog for a file in Explorer and selecting the "Security" tab.
First let's create a small test file in the regular Windows command shell (cmd.exe) and list the NTFS ACL information:
C:\temp>echo this is foo.txt > foo.txt
C:\temp>xcacls foo.txt
C:\temp\foo.txt BUILTIN\Administrators:F
ACTIVE\trentm:F
NT AUTHORITY\SYSTEM:F
BUILTIN\Users:R
This seems reasonable: the "Administrators", "trentm" (that's me), and "SYSTEM" users have full (F) permissions on that file and the "Users" account has read (R) access.
Now let's create a file using one of the cygwin utilities and dump the NTFS ACL information. I'll use tee here, but other tools that create files (like tar, gzip, etc.) will have the same result.
C:\temp>echo this is bar.txt | tee bar.txt
this is bar.txt
C:\temp>xcacls bar.txt
C:\temp\bar.txt ACTIVE\trentm:(special access:)
STANDARD_RIGHTS_ALL
DELETE
READ_CONTROL
WRITE_DAC
WRITE_OWNER
SYNCHRONIZE
STANDARD_RIGHTS_REQUIRED
FILE_GENERIC_READ
FILE_GENERIC_WRITE
FILE_READ_DATA
FILE_WRITE_DATA
FILE_APPEND_DATA
FILE_READ_EA
FILE_WRITE_EA
FILE_READ_ATTRIBUTES
FILE_WRITE_ATTRIBUTES
BUILTIN\Users:(special access:)
READ_CONTROL
SYNCHRONIZE
FILE_GENERIC_READ
FILE_GENERIC_WRITE
FILE_READ_DATA
FILE_WRITE_DATA
FILE_APPEND_DATA
FILE_READ_EA
FILE_WRITE_EA
FILE_READ_ATTRIBUTES
FILE_WRITE_ATTRIBUTES
Everyone:(special access:)
READ_CONTROL
SYNCHRONIZE
FILE_GENERIC_READ
FILE_GENERIC_WRITE
FILE_READ_DATA
FILE_WRITE_DATA
FILE_APPEND_DATA
FILE_READ_EA
FILE_WRITE_EA
FILE_READ_ATTRIBUTES
FILE_WRITE_ATTRIBUTES
I don't want to unnecessarily ring alarm bells because my experience has shown that this usually doesn't cause problems in normal usage of the cygwin tools (we use them heavily here at ActiveState). However, yesterday something happened with respect to NTFS ACLs on my Windows developement machine yesterday such that I no longer have write permissions for files created by the Cygwin tools. I can't help but think that the difference in ACL information for foo.txt and bar.txt in this experiment is part of the problem.
Posts
