Code, Camera, Action

Yesterday Apple updated the digital camera raw support for both Leopard and Snow Leopard. 

That's good news for folks with new Canons or Nikons, as the update adds support for recent cameras.

But Micro Four-Thirds users are still left crying in the wilderness. 

• No support for the Olympus EP-1's ORF raw format
• Still no support for Panasonic RW2 raw in the Lumix LX3, G1, GH1, GF1, etc. -- that's any Panasonic camera released since the fall of 2008
• No support for DNG 1.3 lens corrections, which means there's no good work-around. Adobe DNG Converter can turn these raw formats into proper DNGs. But OSX can't read them.

I'm beginning to get worried. 

The trick with these micro four-thirds cameras is that the camera corrects for lens distortion in the raw files. That helps Olympus and Panasonic save money on lenses, while still producing stellar images. But decoding these raw formats adds a layer of math, which complicates image processing. 

I used to think this was a problem that Apple would fix in Snow Leopard. I hoped the new operating system would add lens corrections to its image processing. But this is the first raw update for Snow Leopard. 

Does Apple ever intend to support these raw formats? 

Maybe they won't -- unless we ask them for it. The Aperture feedback form is the place to let Apple know we want raw support for micro four-thirds cameras:

http://www.apple.com/feedback/aperture.html

In the meantime, there is one alternative. Adobe Lightroom does supports micro four-thirds raw. 

Please, Apple, don't push us away from Aperture and iPhoto. Please add support for micro four-thirds cameras. I like my G1 and I like it raw.

Meanwhile, here's what the update does provide --

Digital Camera Raw Compatibility Update 2.7

This update extends RAW image compatibility for Aperture 2, iPhoto ’08 and iPhoto ’09 for the following cameras:

• Canon EOS-1D Mark IV
• Canon EOS 7D
• Canon PowerShot G11
• Nikon D3S
• Nikon D300S
• Nikon D3000

Loading mentions Retweet

Update: Aperture 3 now supports the Lumix LX3, G1 and GH1 raw formats. However, there are some problems with DNG support for these cameras. The trade-offs in this post are still valid, but before you decide on a DNG workflow, take a look at the work-arounds for getting certain DNGs working in Aperture 3.

Mac OS X and Aperture don't yet support the RW2 raw format produced by recent Panasonic cameras like the DMC-G1 or LX3 or the Olympus E-P1. That leaves Mac users with a couple of options:

• Shoot JPEG. The quality is pretty good, and the G1's automatic settings get it right much of the time.
• Shoot RAW and convert with SilkyPix or RAW Developer, software packages that support RAW from these cameras.  SilkyPix, while it comes with Panasonic cameras, has an unusual interface (export settings get saved in the "cloakroom"). RAW Developer seems great, but is by no means free. Neither program is designed to manage a photo library.
• Buy the latest version of Adobe Lightroom, which handles these latest raw formats just fine
• Shoot RAW and convert to Adobe's DNG format -- which Aperture and iPhoto can read mostly

For now, I am committed to Aperture.  And I prefer to shoot RAW. So if, like me, you prefer to use Aperture to manage your photos, this workflow can help.

As I've written before (in Adobe DNG 1.3 spec now does RAW for the Panasonic Lumix G1), the state of OS X is that for now, DNGs work fine from adapted manual lenses. While the DNG converters have been updated to support shots taken with the Panasonic and Olympus lenses, OS X doesn't understand the new stuff those lenses add.  In practice, this means you can only work with raw shots made with legacy lenses.  One day Aperture and OS X will be updated to understand the new stuff -- either in RW2 format or in DNG. Until that sunny day, here's what I do:

1. With the kit lens (or any other micro 4/3 lens -- if I had others), I shoot JPEG
2. With manual legacy lenses (I have a couple of Voigtlanders), I shoot RAW and convert to DNG
3. Import these DNG and JPEG images into Aperture

Automating Conversion and Import

This may sound complicated, but at download time, it's really not, because I've automated the process.  Here's how you can, too. For that last import step, I use Image Capture to pull the photos off of the camera and call an Automator workflow that does the conversion.  

I've put the code for the the workflow below. It requires two helper applications:

• Adobe DNG converter turns your RW2 files into DNGs. The originals are still there, if you want them.
• exiftool, which compensates for a date bug. Aperture and DNG converter can't agree on the date and time your photos were taken. Exiftool removes date info from the DNGs so that Aperture can do the right thing.

These programs get called by the import workflow, which looks like this in Automator:

Here's the code for the shell script action that does the conversion:

Simply paste that code into a perl shell script workflow box, just like in the screenshot above. For the import action, feel free to choose import into Aperture (like I do) or iPhoto. Either one will work.

Last, save the workflow as an Image Capture plugin, and you'll be all set. Image Capture will let you pick where to download the images, and your new workflow will appear in the list of automatic tasks. (Restart Image Capture if you don't see it right away.)

Or Download the Whole Workflow

Update: Don't want to mess around with creating a workflow to do this? Why not just download mine. Save it in your home directory to ~/Library/Workflows/Applications/Image Capture, where Image Capture will find it. Restart Image Capture if necessary.

So there it is. You'll be shooting full quality when you can, or JPEG when you must -- and not complicating your workflow.

Click here to download:

RW2_to_DNG_to_Aperture.zip (235 KB)

Loading mentions Retweet

Turns out Time Machine is not just for accidentally deleted files — it works great for restoring a Mac from bare metal, too. Even over the network.

James Duncan Davidson’s instructions for using theOSX Leopard install DVD to boot and restore from a Time Machine backup work great if you back up to aUSB or FireWire drive.

But I use Time Machine over the network, so my backup set wasn’t available. Nedospasov had the secret for mounting the network backup share first.

Here’s the whole process:

1. Boot from the installer DVD. Choose “Options…” to get the Utilities menu

2. Utilities > Disk Utility to partition the disk the way you like it

3. Utilities > Restore System from Backup… If your Time Machine backup set appears, then restore away. While there’s a button here to connect to a remote disk, that option wasn’t clickable for me. So we’ll have to mount our network volume manually.

4. Utilities > Terminal Run this command to mount your backup volume manually (substitute your username, password, and so forth):

mount -t afp afp://username:password@hostname/ShareName /Volumes

5. Utilities > Restore System from Backup… and you should now be able to choose your Time Machine backups.

Wait for System Restore to do it’s thing. Over my 100Mb network, it took about four and a half hours to restore 80G. WiFi would require even more patience.

Once the restore is done, you’ll be asked to reboot. And then, ta-da — just about everything will be back just the way it was. A couple of exceptions: Mail will rebuild it’s cache the first time you start it. You’ll have to re-authorize iTunes. And if you use Web Sharing, Apache won’t start because its log directory isn’t restored.

To fix Web Sharing, recreate Apache’s log directory. From Terminal, run this command:

sudo mkdir /private/var/log/apache2

And that’s all. At least for OSX 10.5.3. While Time Machine does not give you a bootable external drive the way that a backup with SuperDuper! might, Time Machine support baked into the OSX install DVD makes this a great option. With Time Machine we get file versioning and disaster recovery. Way to go, Apple!

Loading mentions Retweet

So the Mac OS X 10.5.3 update broke Time Machine for me.

The update did fix Time Machine backups over the network for many people, but not for me. Lots of folks have tried various things to fix the problem (like standing on their head, reformatting, fingering prayer beads). Here’s what fixed Time Machine over AFPfor me.

I had a setup that was backing up perfectly on 10.5.2.

• MacBook connected to wifi network
• Mac Mini with external drive
• A shared folder on the Mini’s external drive (called “backups”) that was the Time Machine backup target. This folder contains the sparse bundle holds all the MacBook’s backups
• Access to the Mini’s “backups” share was through a .Mac login
  • Here’s how I knew that: Browsing to Mini in Finder shows “Connected as: blahblah@mac.com” in the gray bar
  • And the local usernames on the two machines were different (the same .Mac login accessed harold on the MacBook and freddy on the Mini)

After the 10.5.3 update, Time Machine said “The backup volume could not be mounted” — again and again. Console’s system log showed backupd errors 18 and sometimes 19.

Worse, I couldn’t choose “backups” as the destination for Time Machine. As it turns out, there are two problems with this setup that Leopard 10.5.3 doesn’t like — even though 10.5 through 10.5.2 were fine with it.

1. 10.5.3 Time Machine now apparently can’t use a share accessed by a .mac login
   • At least when they point to different local usernames. Let us know in the comments if it works for you with .mac at all
2. Permissions on previous sparse bundle backups now seem to confuse Time Machine
   • Again, this is likely due to the .mac credentials

I was able to fix the first of these problems and work around the second both problems, which got Time Machine backing up over the network.

I am so far unable to access previous Time Machine backups, though, so I’m still experimenting with the second.

Fixing Problem Number 1 – Connect as you you, not .mac you

1. On MacBook, start System Preferences, and choose Time Machine
2. “Change disk”, and select “None”
   • This turns Time Machine backups off for now.
   • Some folks say you need to reboot now
3. On MacBook unmount any of Mini’s shares
4. Over on Mini, share your backup share with both your user and Administrators; both need Read & Write access
5. Back on MacBook, connect to “backups” on Mini as your username
   • The .Mac user is the default, so you’ll probably need to disconnect, then connect as yourself
6. On MacBook, start System Preferences, and choose Time Machine
7. “Change disk”, and select “Backups (mini)”
   • “Backups” should be in the list now. If not, reboot and start from step 4.
8. Enter your username and password. With this, Time Machine will be able to mount your share automatically when it’s ready to back up
9. Try to “Back up now” in Time Machine

If it works, congratulations, you’re done. For me, it didn’t — probably because the previous .mac login connected as different users on Mini and MacBook. Since you’re not likely to have such a bizzarro-world setup, you’re probably home free. Let us know in the comments.

Fixing Problem Number 2 – sparsebundle permissions

Because of my .mac username hijinks (the same .Mac login accessed harold on the MacBook and freddy on the Mini), I still couldn’t back up.

Console told me I was now getting backupd error 18 in system.log:

/System/Library/CoreServices/backupd: Volume at path /Volumes/Backups-1 does not appear to be the correct backup volume for this computer. (Cookies do not match)
/System/Library/CoreServices/backupd: Backup failed with error: 18

In theory, changing permissions on the sparsebundle to MacBook’s user ought to work. Only it didn’t for me.

Instead, I moved the sparsebundle on Mini to another folder. From there, MacBook’s Time Machine backed up just fine, though any backups made under 10.5.2 or earlier are lost.

Update 1 Jun: It does work, but I had to change two things to get access to the old backups.

1. Give the local user read and write permissions on the sparsebundle. Use Finder on the Mini to do this, as ACLs are involved. It will take a long time.
2. From Terminal on Mini, change ownership of the sparsebundle to the local user (use your username in place of “eric” and your sparsebundle name in place of “greatnorthern…”):

sudo chown -R eric greatnorthern_0017f2f16a4e.sparsebundle

After this, backups over wifi worked — though the first backup took a very very long time preparing.

The good news? 10.5.3 is much smarter about network backups. Time Machine will mount the share automatically when it’s time to back up — or when you enter Time Machine. For a laptop, Time Machine backups are now truly seamless.

If only the upgrade was painless!

6 Comments on the Original Post

Loading mentions Retweet

Both Apache 2.2 and PHP 5 come with every installation of Mac OS X 10.5 Leopard. MySQL does not. There are a couple of tricks for installing and/or turning them on —and thereby attaining Development Happiness.

Here’s how we configure our development machines at the studio, where our PHP work is primarily Drupaland we use several virtual hosts.

Install MySQL

Not hard, but a little long for this post. Take a look at our instructions for installing MySQL on Leopard.

Configure Apache for PHP and Virtual Hosts

Though Leopard ships with PHP, it’s not turned on by default. Let’s change that, and also turn on the virtual hosts that we’ll use for our various Drupal installs.

• Edit Apache’s configuration file to make the following two changes:
  

  
  
  sudo vi /etc/apache2/httpd.conf
  
  

• Uncomment line 114 so it looks like this:
  

  
  
  LoadModule php5_module        libexec/apache2/libphp5.so
  
  

• Uncomment line 461 so it looks like this:
  

  
  
  Include /private/etc/apache2/extra/httpd-vhosts.conf
  
  

Add your Virtual Hosts

Edit /etc/apache2/extra/httpd-vhosts.conf to add your configuration. See our sample configuration for ideas.

NameVirtualHost *:80
<VirtualHost *:80>
    ServerAdmin eric@localhost
    DocumentRoot "/Users/eric/Sites/agc"
    ServerName agc.local
    RewriteLog /private/var/log/apache2/rewrite_log
    RewriteLogLevel 0
</VirtualHost>

A couple items of note:

• For development, we put our virtual hosts under users’ ~/Sites folder. That way there are no permissions issues to prevent Apache from serving the code — or from deploying fresh.
• You’ll also have to put agc.local into DNS.

Add the new virtual hostnames to /etc/hosts

Edit /etc/hosts to add a couple of nicknames for localhost:

127.0.0.1 localhost agc.local giving.local givingedit.local miyo.local

Start Apache

• Start System Preferences from the Apple menu.
• Go to the Sharing tab
• Click the checkbox by “Web Sharing”

Done. Let’s test everything by starting Safari and visiting http://agc.local

Loading mentions Retweet

MySQL now provides compiled binaries for Mac OSX 10.5 Leopard. But the startup applet doesn’t work properly yet on Leopard.

So here’s how to install MySQL on Leopard and start it using launchd — the standard Leopard way. These instructions should work for both Tiger (10.4) and Leopard (10.5).

• Download MySQL for your OS and architecture (PowerPC or Intel; 10.4 Tiger or 10.5 Leopard)
• Install MySQL from the package in the dmg — something like mysql-5.0.51a-osx10.5-x86_64.pkg
• DO NOT install the startup item package — it doesn’t work on Leopard (as of version 5.0.51a).
• DO NOT install the preference pane — it doesn’t work on Leopard (as of version 5.0.51a, though a fix is in the works).

Starting MySQL

Since neither the preference pane nor startup item work on Leopard, let’s use launchd to start MySQL instead.

• Copy com.mysql.mysqld.plist to /Library/LaunchDaemons
• To start MySQL, use the launchctl command:
  

  
  
  sudo launchctl load -w /Library/LaunchDaemons/com.mysql.mysqld.plist
  
  

• Stop mysqld with an unload
  

  
  
  sudo launchctl unload -w /Library/LaunchDaemons/com.mysql.mysqld.plist
  
  

The -w in either of these commands leaves mysqld running (and restarts it as needed) until you turn it off with the other launchctl command.

For convenience, you may want to create an alias for these commands in .bashrc

alias start_mysql='sudo launchctl load -w /Library/LaunchDaemons/com.mysql.mysqld.plist'
alias stop_mysql='sudo launchctl unload -w /Library/LaunchDaemons/com.mysql.mysqld.plist'

One other convenience — put mysql and mysqldump into your path:

cd /usr/local
sudo ln -s mysql-5.0.51a-osx10.5-x86_64 mysql
cd bin
sudo ln -s ../mysql/bin/mysql .
sudo ln -s ../mysql/bin/mysqldump .

Done. So we’re ready to fire it up and connect:

sudo launchctl load -w /Library/LaunchDaemons/com.mysql.mysqld.plist
mysql -u root -p

Loading mentions Retweet

So you want to use Git for distributed version control. For Mac OS X 10.5 there are a couple of options for installing Git.

• Fink packages an almost up-to-date version, named simply git
• MacPorts packages an up-to-date version asgit-core

Either of these methods, though, installs lots of dependencies. Which requires lots of compile time. And lots of disk space — especially for the MacPorts version.

So I prefer to compile Git myself — and use Fink for dependencies. Here’s how I do it.

Install Fink by following Fink’s installation instructions.

fink install gettext-tools expat1-shlibs

From here the instructions in Git’s INSTALL file work just fine:

curl -O http://kernel.org/pub/software/scm/git/git-1.5.4.5.tar.gz
tar zxf git-1.5.4.5.tar.gz
cd git-1.5.4.5
make prefix=/usr/local all 
sudo make prefix=/usr/local install

Now for the manpages.

Rather than build the docs ourselves (which requires asciidoc and a host of dependencies), I prefer to download them precompiled from Git and manually copy them into /usr/local/share/man.

curl -O http://kernel.org/pub/software/scm/git/git-manpages-1.5.4.5.tar.gz
sudo tar zxf git-manpages-1.5.4.5.tar.gz  -C /usr/local/share/man

(Replace the version numbers here with what’s right for the version of Git you’re installing.)

Now we’re ready to try this out, we’ll ask what version we just installed, then show the manpage:

git --version
man git

Done. Happy versioning!

Update

Sean Santry has an even simpler way to avoid the gettext-tools dependency — by telling make that we don’t have msgfmt at all.

NO_MSGFMT=yes make prefix=/usr/local all
sudo make prefix=/usr/local install

Using this tip you can build Git without Fink altogether!

Comments on the Original Post

You can also just install a complete OS X package from http://code.google.com/p/git-osx-installer/. It’s only 3.6MB and spares you the manual install

Pieter de Bie 4 Apr 2008, 10:31 AM

Loading mentions Retweet

Leopard does not (yet?) backup to a disk attached to an Airport base station, to lots of people’s concern. But OS X version 10.5.1 will backup over the network — to another Mac. It works just fine over wired or wifi networks, so it works for MacBooks.

Initial setup is easy.

On the host Mac: Plug in your external drive. Turn on File Sharing from System Preferences (AFP, the default, is what you want). Share a directory on your external drive where you’ll store the backups.

On the Mac you want to back up: Mount the backup share. Then go to Time Machine in System Preferences, and turn it on. When it asks you for a backup destination, select your backup share.

Each time you backup…

Time Machine then needs two things before it will backup your MacBook:

1. Plug the laptop into a power supply. Time Machine won’t start a backup if you’re running on battery power.
2. Mount the share that holds your backups. (Double-click on it in Finder.)
3. Done. Time Machine will back up on schedule, so you can keep working. But if you’re anxious, you can also start the backup by picking “Back Up Now” from the Time Machine icon.

I find these two steps to be much more convenient than plugging the external drive into the laptop, and I hope you will, too.

Update 31 May 2008: Since Leopard 10.5.3, Time Machine no longer requires any of this — it mounts backup shares when needed, so you don’t have to do anything. Once you get Time Machine working again, that is. If you’re having troubles since the 10.5.3 update, readA Fix for Time Machine on OSX 10.5.3 over AFP for .mac Users, a how-to that will get you going again.

Loading mentions Retweet