Difference between revisions of "SMILA/Documentation/Binary Storage"

Revision as of 08:01, 26 June 2009

Overview

Binary Storage Service provides an easy way to store / access binary data documents.

Client components access the Binary Storage Service for persisting binary data (attachments) into the binary storage. The binary data are identified by a unique key / identifier as a String data type.

Configuration

Currently, Binary Storage is able to run and store data into a hierarchical structure (uses deterministically calculation based on the hash id passed by the client component, like the ID passed from the Blackboard Service) and flat structure.

To configure the needed persistence storage structure, simply edit the /configuration/org.eclipse.smila.binarystorage/BinaryStorageConfiguration.xml file and change the value of implementationClass attribute to the one of the following available possibilities:

• org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOHierarchicalManager
• org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOFlatManager

In the next future there will be other available managers that provides the appropriate implementation for Binary Storage persistence structure.

The Binary Storage Configuration will be changed soon.

Configuration example

<BinaryStorageConfiguration xmlns="http://www.eclipse.org/smila/binarystorage" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:noNamespaceSchemaLocation="schemas/BinaryStorageConfiguration.xsd"
  name="default1"
  provider="file"
  mountPoint="default"
  implementationClass="org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOHierarchicalManager" 
  tempFileName="dummy.dat"
  pathDepth="2"
/>

Attributes explained:

• name - configuration alias
• implementationClass - binary storage implementation manager class
• tempFileName - binary storage temporary file name
• pathDepth - the depth of the folder structure (Note: If not set the default value is 2.)

API

void store(String id, InputStream stream);
void store(String id, byte[] blob)
byte[] fetchAsByte(String id)
InputStream fetchAsStream(String id)
void remove(String id)
int fetchSize(String id)

API Usage

BinaryStorageService binaryStorageService = ...; // Obtain the binary storage service
String attacmentIdKey = ...; // Unique ID
byte[] dataRecord = ... // Build record
 
// Store the binary record as byte array
binaryStorageService.store(attacmentIdKey, dataRecord);
 
// Fetch record as stream
final InputStream stream = binaryStorageService.fetchAsStream(attacmentIdKey);
 
// Define new ID and store new data from the first record's input stream
final String newAttacmentIdKey = attacmentIdKey + "0000";
binaryStorageService.store(newAttacmentIdKey, stream);
stream.close();
 
// Fetch the newest record as array of byte 
final byte[] recordByte = binaryStorageService.fetchAsByte(newAttacmentIdKey);
 
// The two records must have the same size
final int sizeInitial = _binaryStorageService.fetchSize(attacmentIdKey);
final int sizeFinal = binaryStorageService.fetchSize(newAttacmentIdKey);
 
// Remove the two records
binaryStorageService.remove(attacmentIdKey);
binaryStorageService.remove(newAttacmentIdKey);

Implementation Limitations and Notes

Both I/O implementations use the id in the path name of the persisted file. As a consequence

• the characters [;/\:] are not allowed in the id
• too long ids will exceed the platform specific supported max. path length. this will result in a not so obvious error message, that saving failed.

ATM neither limitation this is a problem because the id is a hash made from letters and numbers.

IOHierarchicalManager

The calculation of the path is simply to split the first part of the id into substrings of length 2 and make thees parent folders having the name of the id.

Care must be taken that the folder depth is not too high, because this results in a lot of folders quenching the performance of the file system.

IOFlatManager

If u only have a few files or ur underlying file system is very good inhandling lots of files per folder then go with this manager.