Author: Ben Martin
23hq://username/upload.
There are already many tools around that support uploading photos. For instance, F-Spot can export to these sites and Kflickr can upload by dragging files onto its window. However, libferris includes features that take advantage of metadata that make it an attractive alternative to these methods.
When you upload an image using libferris, the application stores metadata about the original local file that records what remote photo-id and server you uploaded to. By using the metadata with the desktop index and search, you can later find the original local image file from a flickr or 23hq page URL.
You may use the remote site’s tagging support to explicitly send through local metadata about the image. For example, you could send the geotags of the local image, its MD5 checksum, or F-Spot tags to the Web photo site. Support for viewing F-Spot tags through libferris as metadata is also a new feature in the 1.0.106 release.
Libferris also offers the standard features one expects when uploading a digital photo: setting default view permissions and scaling the image that is to be uploaded to preserve your monthly bandwidth quota.
Setting up authentication
Libferris’s authentication style is the flickr desktop auth scenario with a minor twist — that being that you’ll have to obtain an API key and secret from your Web photo site. This procedure enhances the security of your account and ensures that you accept the terms and conditions of the photo site. You can obtain an API key by using steps one and two of the flickr desktop auth page. 23hq recommends using the flickr API key or just making one up. Once you have the API key and API secret, place them in ~/.ferris/flickr-api-key.txt and ~/.ferris/flickr-api-secret.txt respectively. Users of 23hq should substitute “23hq” in the file names where “flickr” appears.
You will find many graphical configuration tools in the cc/capplets directory of the libferris distribution. All of these tools begin with the ferris-capplet prefix. With the API key in place, run the cc/capplets/auth/ferris-capplet-auth tool. Clicking on the authenticate button will bring up a dialog with the URL to visit to log in to the photo service and authenticate the API key to access your account. The dialog offers to run Firefox for you on the URL, or you can just copy it to a browser of your choice to complete the browser section of the authentication. After you have authenticated the API key in your browser, return and OK the dialog to complete the transaction. At this stage the capplet will show Status OK and your username.
You might also want to check the settings tab at this stage. The lower part of the dialog is common to most image uploading tools. You can set the default viewer permissions and resize an image to have a largest dimension of 800, 1024, 1920, etc. Using a single “largest dimension” allows you to handle both landscape and portrait aspects with the same resizing setting.
The two strings at the top of the settings tab allow you to tell libferris which metadata from the local file to upload as tags on the remote image file. Libferris handles metadata through an extension of the kernel’s extended attributes (EA) interface. Extended attributes mean that you can attach to each file or directory key-value pairs that expose metadata about files and directories. You can consider EA and metadata to be the same thing in the context of this article. Using a regular expression allows you to quickly select groups of metadata to upload. For example, f-spot:.* will upload any tags you have assigned the local image using the F-Spot image viewer.
The “is present” regular expression means that if there is an extended attribute on the file and its value is not zero or false, set a tag on the remote file for the key part of the extended attribute. The “and Value” regular expression sets the extended attribute key and its value as the tag on the remote image.
For an example of the “is present” regex, consider having tagged your image files in F-Spot to indicate who is present in each image and the image location. Say one of your friends is tagged fred. If you use F-Spot and attach the fred tag to an image, libferris will see an extended attribute (metadata) for that image with the key of “f-spot:fred” and the value of “1”. Setting the “is present” regex to f-spot:fred will set a tag of f-spot:fred on the uploaded image whenever the local image has the fred tag associated in F-Spot. A handy starting point for your “is present” regex is (emblem:has-.*|fspot:.*). This expression makes sure that any F-Spot tags for the image are sent through as tags on the remote image. The emblem:has- part of the regular expression makes sure that any libferris emblems attached will be sent as tags on the remote image.
The libferris notion of emblems is similar to what is commonly called tags these days, with a few differences. For example, emblems can be partially ordered in libferris, and a single emblem-file association can be made by many different agents to allow computer-assisted automatic emblem assignments. But for this article, you can think of emblems as tags.
An example of the “and value” regex would be setting it to md5. In this case the remote image would have a tag like md5=d3b07384d113edec49eaa6238ad5ff00, where the actual md5 of the local image file is to the right of the equals sign. A handy starting point for your “and value” regex is (size|md5|emblem:id-.*|fspot:.*|latitude|longitude). This makes sure that the size and md5 of the original image are sent through, as well as any F-Spot tags for the image. The emblem:id- part sends the immutable libferris emblem number as a tag on the remote image. This number is useless for other people, but you can use it to find other photos in your local photo storage that have the same libferris emblems. The latitude and longitude in the regex make sure that your local geotagged image location is sent to the server. I described setting up geotags in a previous Linux.com article.
Notice that both of the above regular expressions mention the emblem: namespace of libferris. The “and Value” regular expression sends through the unique numerical identifier for an emblem, and the “is Present” regular expression sends through the text label of the emblem. For example, you might create a new emblem “Central station” in libferris. Libferris will automatically assign an immutable unique (among all emblems you have) number for this emblem — say 17. A file that has this emblem attached will have two extended attributes: emblem:has-Central station and emblem:id-17. If you are ordering your emblems in libferris then you might have a Central station emblem for many towns, it is quite common for each towns Central station emblem to just have a label “Central station” because the town it is in (its parent emblem) will let you know if you are talking about the central station of Sydney or Berlin. Though the label might be exactly the same, each central station emblem will have a unique numerical identifier. This allows you and libferris to be able to tell which central station emblem you have attached to a file. By sending through both the emblem label (emblem:has-Central station) and its unique identifier emblem:id-17 to the webphoto site you allow users to know what the emblem was (by its label) and yourself to know exactly which emblem you tagged the image with regardless of how many emblems have the same label. –>
Uploading an image
All authenticated accounts appear under the URL scheme webphotos:// in libferris. Flickr and 23hq are subdirectories of webphotos. Each of these image server subdirectories has in turn subdirectories for each user account you have authenticated, in order to support image services that offer the ability to have multiple registered user accounts. There are also shortcuts flickr:// and 23hq://, which link directly to the respective subdirectories of webphotos.
You can use libferris’s image uploading features in many ways: from the ferriscp or gfcp clients, using the ferrisfs FUSE filesystem with any application, or using drag and drop into the ego file manager when you’re viewing the upload directory. The ferriscp client is a command-line clone of the coreutils cp(1) tool. The gfcp client is like ferriscp but with a GTK2 user interface.
Both the ferriscp and gfcp clients use the same arguments: the source file(s) and the destination directory, much like the regular cp(1) command. Either of the below commands will upload the image to 23hq for the authenticated user name monkeyiq:
$ ferriscp IMG_0355.JPG 23hq://monkeyiq/upload $ gfcp IMG_0355.JPG 23hq://monkeyiq/upload $ ferriscp --target-directory 23hq://monkeyiq/upload *.jpg
The last command form is handy if you want to drag and drop your file paths from another application, such as a Web browser, onto the console to complete the command.
Copying files to the upload directories will respect your default permissions and cause libferris to scale the image to upload if you have set a desired maximal image dimension.
The auth tab will show your user name when you have successfully authorized libferris to access your web photo account. Also, listing the URL 23hq:// will show a subdirectory for any authenticated user accounts.
If you’re using the ego file manager 0.16.0 or above, you can drag and drop images to upload them. For example, view the 23hq://monkeyiq/upload directory and drag a file from F-Spot onto the file list to upload it.
Using ferriscp, gfcp, or ego to upload will also cause your nominated metadata to be sent to the image server, and will attach extra metadata to the original local image file showing what the remote photo-id is and which server you uploaded to.
Finding the originals again
If you use a free Web photo account you may be limited to downloading a scaled version of the photo you uploaded. Also, if you have set images to be scaled before uploading them, you will at times want to find the original photo image on your PC when viewing an image on the Web photo server.
The 1.0.106 release of libferris includes a special client to use to find the original version of any image you have uploaded to a Web photo server with libferris. The ferris-webphoto-remote-url-to-eaindex-predicate client takes the URL of a page that shows one or more photos and returns a desktop search predicate that can be used directly with the feaindexquery command-line tool to find the original images. Note that although the command name is long it can be obtained with fer, tab, -w, tab, and then a choice among two commands which spell out their action. I’ll refer to this remote-url client simply as eaindex-predicate to save space. The --photo-url option makes the eaindex-predicate client assume that you are passing a URL that features a single image file. The --album-url argument indicates that the URL has many photos on it — for example the default page for a user account — and makes a predicate to find all the images on the page.
$ ferris-webphoto-remote-url-to-eaindex-predicate --photo-url
http://www.23hq.com/monkeyiq/photo/1766285
(23hq-photo-id==1766285)
$ ferris-webphoto-remote-url-to-eaindex-predicate --album-url
http://www.23hq.com/monkeyiq
(|(23hq-photo-id==1766285)(23hq-photo-id==1815816)...))
You can execute the eaindex-predicate client and place its output as the command-line argument to feaindexquery. Having the eaindex-predicate client return a predicate like this allows you to easily build a more complex query. For example, surrounding the output of eaindex-predicate with (&(mtime>=begin this year)...), replacing the ellipsis with the eaindex-predicate output, will find only matching Web photos that were modified this year. You have to surround the eaindex-predicate call with double quotes to stop bash from trying to interpret the () characters. The eaq:// filesystem prefix presents the result of a query as a virtual filesystem. The ferriscp command uses the eaq:// prefix to run a query directly and operate on the result of that query as though it were the argument. For example, to copy foo.jpg to /new-dvdrw:
$ feaindexquery
"$(ferris-webphoto-remote-url-to-eaindex-predicate --photo-url
http://www.23hq.com/monkeyiq/photo/1766285)"
Found 1 match at the following locations:
file://home/saruchin/dpix/foo.jpg
$ ferrisls -lh
"eaq://$(ferris-webphoto-remote-url-to-eaindex-predicate --photo-url
http://www.23hq.com/monkeyiq/photo/1766285)"
.... foo.jpg
$ ferriscp
"eaq://$(ferris-webphoto-remote-url-to-eaindex-predicate --photo-url
http://www.23hq.com/monkeyiq/photo/1766285)"
/new-dvdrw
If you have specified that you want to upload the md5 checksum of your images as “and value” regex tags, then you can also use the md5 tag value from the Web photo service to find the local image again. Just copy the md5 value from the Web site and query for files matching that checksum directly. For example, the image here would have the second query below.
$ feaindexquery "(md5==8316eb6471e946f27975321c612e7b30)"
This query relies on your index including the md5 checksum for every file. As this can be fairly computationally expensive, it may not be enabled by default. For fairly static filesystems like local digital photo repositories, you can create a federated index with libferris and only include the md5 for your digital photo filesystem index. Federated indexing is explained in the Federated Desktop and File Server Search with libferris article.
For more information on desktop search with libferris, see the Linux Journal articles here and here.
Web sites that provide Web services can offer interesting virtual filesystems when mounted. The ability for the filesystem itself to record metadata for operations such as remembering where a source image was uploaded and the remote site’s identifier for that file allows application developers to bring together information easily at a later time.
Although libferris has been able to mount HTTP and FTP as filesystems for a long time, it has just begun exploring the potential of mounting Web services. Exposing information at the filesystem level, either using libferris or through FUSE, makes Web service sites reachable from bash and command-line tools.
