Difference between revisions of "SMILA/Documentation/Importing/CompoundExtractorService"

Revision as of 05:27, 22 February 2012

CompoundExtractor Service

Interface: org.eclipse.smila.importing.CompoundExtractor

A CompoundExtractor service provides two kinds of methods:

• check if an object's filename, URL or mimetype idenfifies it as a compound object that can be extracted by the service.
• extract the compound: Given an InputStream with the compound content produce records for the elements.

The element records can contain the following attributes:

• fileName: the complete name of the entry in the compound object, usually something like a filesystem path
• isCompound: set to true if the element is a supported compound object itself.
• size: uncompressed size of the element
• time: last modification timestamp, as a datetime value.
• compressedSize: compressed size of the element
• comment: a comment for the element in the compound
• isRootCompound: set to true if the record describes the processed compound object itself.
• compounds: a sequence of the compound files to look into to reach this element. For example, if the compound /data/compound.zip contains a file archived/subcompound.zip which contain a file x.html, the compounds list for x.html would be:

  [/data/compound.zip, archived/subcompound.zip]

Not all attributes need to be set for all compound types.

Implementations

SimpleCompoundExtractorService

Bundle: org.eclipse.smila.importing.compounds.simple

This extractor service uses the classes provided by the JDK's java.util.zip package to extract compound objects. This means that it can currently support ZIP files and GZ files (not TAR.GZ, though).

Supported Mimetypes:

• application/zip
• application/x-gunzip
• application/x-gzip

If the mimetype is not provided by the caller at all or it is only application/octet-stream it uses the current MimeType Identifier service to recognize the real mimetype from the filename extension.

The compound types are treated differently:

• For ZIP files, it creates one record for the ZIP file itself and one record for each contained element.
• For GZ files, it creates one record with the original filename of the GZ file, but the uncompressed content, because technically GZ files are not real compounds, but just compressed single files.

Configuration

The simple extractor service can be configured by means of a properties file configuration/org.eclipse.smila.importing.compounds.simple/extractor.properties.

The configuration properties are as follows:

• zip.encoding the encoding to use when extracting ZIP files that do not use UTF-8 (default: UTF-8)
  • example: zip.encoding=CP850
• tmp.dir the temporary directory to extract compounds to, per default a directory named org.eclipse.smila.importing.compounds.simple is created in the user's temporary folder (e.g. on Windows 7 something like C:\Users\<username>\AppData\Local\Temp\org.eclipse.smila.importing.compounds.simple).
  • example: tmp.dir=/temp/SMILA.compound.extractor/

CommonsCompressCompoundExtractorService

Not yet in SMILA! As soon as we are allowed to use the Apache Commons Compress library we will provide a extractor service based on this library and then be able to support more compound types.

Bundle: org.eclipse.smila.importing.compounds.compress

Supported Compound Formats:

• Archives
  • application/zip
  • application/x-tar
  • application/us-tar
  • application/x-cpio
  • application/x-bcpio
  • application/x-sv4cpio
  • application/java-archive
• Compressions
  • application/x-bzip
  • application/x-gtar
  • application/x-gzip
  • application/x-gunzip

Configuration

The simple extractor service can be configured by means of a properties file configuration/org.eclipse.smila.importing.compounds.compress/extractor.properties.

The configuration properties are as follows:

• zip.encoding the encoding to use when extracting ZIP files that do not use UTF-8 (default: UTF-8)
  • example: zip.encoding=CP850
• tmp.dir the temporary directory to extract compounds to, per default a directory named org.eclipse.smila.importing.compounds.simple is created in the user's temporary folder (e.g. on Windows 7 something like C:\Users\<username>\AppData\Local\Temp\org.eclipse.smila.importing.compounds.simple).
  • example: tmp.dir=/temp/SMILA.compound.extractor/