SMILA/Documentation/Binary Storage
Overview
Binary Storage Service provides an easy way to store / access binary data documents.
Client components access the Binary Storage Service for persisting binary data (attachments) into the binary storage. The binary data are identified by a unique key / identifier as a String data type.
Configuration
Currently, Binary Storage is able to run and store data into a hierarchical structure (uses deterministically calculation based on the hash id passed by the client component, like the ID passed from the Blackboard Service) and flat structure.
To configure the needed persistence storage structure, simply edit the /configuration/org.eclipse.smila.binarystorage/BinaryStorageConfiguration.xml file and change the value of implementationClass attribute to the one of the following available possibilities:
- org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOHierarchicalManager
- org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOFlatManager
In the next future there will be other available managers that provides the appropriate implementation for Binary Storage persistence structure.
The Binary Storage Configuration will be changed soon.
Configuration example
<BinaryStorageConfiguration xmlns="http://www.eclipse.org/smila/binarystorage" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="schemas/BinaryStorageConfiguration.xsd" name="default1" provider="file" mountPoint="default" implementationClass="org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOHierarchicalManager" tempFileName="dummy.dat" pathDepth="2" />
Attributes explained:
- name - configuration alias
- implementationClass - binary storage implementation manager class
- tempFileName - binary storage temporary file name
- pathDepth - the depth of the folder structure (Note: If not set the default value is 2.)
API
void store(String id, InputStream stream); void store(String id, byte[] blob) byte[] fetchAsByte(String id) InputStream fetchAsStream(String id) void remove(String id) int fetchSize(String id)
API Usage
BinaryStorageService binaryStorageService = ...; // Obtain the binary storage service String attacmentIdKey = ...; // Unique ID byte[] dataRecord = ... // Build record // Store the binary record as byte array binaryStorageService.store(attacmentIdKey, dataRecord); // Fetch record as stream final InputStream stream = binaryStorageService.fetchAsStream(attacmentIdKey); // Define new ID and store new data from the first record's input stream final String newAttacmentIdKey = attacmentIdKey + "0000"; binaryStorageService.store(newAttacmentIdKey, stream); stream.close(); // Fetch the newest record as array of byte final byte[] recordByte = binaryStorageService.fetchAsByte(newAttacmentIdKey); // The two records must have the same size final int sizeInitial = _binaryStorageService.fetchSize(attacmentIdKey); final int sizeFinal = binaryStorageService.fetchSize(newAttacmentIdKey); // Remove the two records binaryStorageService.remove(attacmentIdKey); binaryStorageService.remove(newAttacmentIdKey);
Implementations
eclipse.smila.binarystorage.persistence.io
Both I/O implementations use the id in the path name of the persisted file. As a consequence
- the characters [;/\:] are not allowed in the id
- too long ids will exceed the platform specific supported max. path length. this will result in a not so obvious error message, that saving failed.
ATM neither limitation this is a problem because the id is a hash made from letters and numbers.
IOHierarchicalManager
The calculation of the path is simply to split the first part of the id into substrings of length 2 and make thees parent folders having the name of the id.
Care must be taken that the folder depth is not too high, because this results in a lot of folders quenching the performance of the file system.
IOFlatManager
If u only have a few files or ur underlying file system is very good inhandling lots of files per folder then go with this manager.
eclipse.smila.binarystorage.persistence.efs
This implementation is based on EFS.
eclipse.smila.binarystorage.persistence.jpa
This implementation uses eclipseLink JPA to store binary objects in an apache derby database named BinaryStorage. The data is stored in table BINARY_OBJECTS:
| Column | Type | Description |
|---|---|---|
| ID | VARCHAR | the Id of the binary object |
| BIN_OBJECT | BLOB | the binary object |
Configuration
In order to use this implementation set parameter implementationClass in file org.eclipse.smila.binarystorage.impl/BinaryStorageConfiguration.xml:
implementationClass="org.eclipse.smila.binarystorage.persistence.jpa.JPABinaryPersistence"
All other configuration options of file org.eclipse.smila.binarystorage.impl/BinaryStorageConfiguration.xml are ignored !
The other configuration needed is a typicall eclipseLink configuration property file. Therin you can specify settings for logging, database connection settings. For more information please refer to the eclipseLink documentation [[1]]. The configuration is located at configuration/org.eclipse.smila.binarystorage.persistence.jpa/persistence.properties.
# EclipseLink properties eclipselink.logging.level=INFO eclipselink.target-server=None eclipselink.target-database=org.eclipse.persistence.platform.database.DerbyPlatform eclipselink.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver eclipselink.jdbc.url=jdbc:derby:workspace/.metadata/.plugins/org.eclipse.smila.binarystorage.persistence.jpa/binarystorage;create=true eclipselink.jdbc.password=smila eclipselink.jdbc.user=smila eclipselink.ddl-generation=create-tables
After starting Smila for the first time, the DDL generation setting will print out some nasty warnings, complaining that it can't create some tables. These warnings are not critical. You can get rid of them by setting eclipselink.ddl-generation=none, but only after Smila was started at least once (and the tables were created).
Limitations
At the moment it is necessary to import all packages containing JDBCDriver classes in org.eclipse.smila.connectivity.deltaindexing.jpa.impl. So for changing from derby to another database it is not sufficient to change the configuration in persistence.properties, you also have to add import package statementsv for the JDBC driver to use to your bundles manifest. This will hopefully be changed with the next release of eclipseLink.