Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "SMILA/Documentation/Binary Storage"
m (→Overview: added category)
|Line 1:||Line 1:|
== Overview ==
== Overview ==
Revision as of 01:56, 17 October 2009
- 1 Overview
- 2 Configuration
- 3 Configuration example
- 4 API
- 5 API Usage
- 6 Implementations
Binary Storage Service provides an easy way to store / access binary data documents.
Client components access the Binary Storage Service for persisting binary data (attachments) into the binary storage. The binary data are identified by a unique key / identifier as a String data type.
Currently, Binary Storage is able to run and store data into a hierarchical structure (uses deterministically calculation based on the hash id passed by the client component, like the ID passed from the Blackboard Service) and flat structure.
To configure the needed persistence storage structure, simply edit the /configuration/org.eclipse.smila.binarystorage/BinaryStorageConfiguration.xml file and change the value of implementationClass attribute to the one of the following available possibilities:
In the next future there will be other available managers that provides the appropriate implementation for Binary Storage persistence structure.
The Binary Storage Configuration will be changed soon.
<BinaryStorageConfiguration xmlns="http://www.eclipse.org/smila/binarystorage" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="schemas/BinaryStorageConfiguration.xsd" name="default1" provider="file" mountPoint="default" implementationClass="org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOHierarchicalManager" tempFileName="dummy.dat" pathDepth="2" />
- name - configuration alias
- implementationClass - binary storage implementation manager class
- tempFileName - binary storage temporary file name
- pathDepth - the depth of the folder structure (Note: If not set the default value is 2.)
void store(String id, InputStream stream); void store(String id, byte blob) byte fetchAsByte(String id) InputStream fetchAsStream(String id) void remove(String id) int fetchSize(String id)
BinaryStorageService binaryStorageService = ...; // Obtain the binary storage service String attacmentIdKey = ...; // Unique ID byte dataRecord = ... // Build record // Store the binary record as byte array binaryStorageService.store(attacmentIdKey, dataRecord); // Fetch record as stream final InputStream stream = binaryStorageService.fetchAsStream(attacmentIdKey); // Define new ID and store new data from the first record's input stream final String newAttacmentIdKey = attacmentIdKey + "0000"; binaryStorageService.store(newAttacmentIdKey, stream); stream.close(); // Fetch the newest record as array of byte final byte recordByte = binaryStorageService.fetchAsByte(newAttacmentIdKey); // The two records must have the same size final int sizeInitial = _binaryStorageService.fetchSize(attacmentIdKey); final int sizeFinal = binaryStorageService.fetchSize(newAttacmentIdKey); // Remove the two records binaryStorageService.remove(attacmentIdKey); binaryStorageService.remove(newAttacmentIdKey);
Both I/O implementations use the id in the path name of the persisted file. As a consequence
- the characters [;/\:] are not allowed in the id
- too long ids will exceed the platform specific supported max. path length. this will result in a not so obvious error message, that saving failed.
ATM neither limitation this is a problem because the id is a hash made from letters and numbers.
The calculation of the path is simply to split the first part of the id into substrings of length 2 and make thees parent folders having the name of the id.
Care must be taken that the folder depth is not too high, because this results in a lot of folders quenching the performance of the file system.
If u only have a few files or ur underlying file system is very good inhandling lots of files per folder then go with this manager.
This implementation is based on EFS.
This implementation uses eclipseLink JPA to store binary objects in an apache derby database named BinaryStorage. The data is stored in table BINARY_OBJECTS:
|ID||VARCHAR||the Id of the binary object|
|BIN_OBJECT||BLOB||the binary object|
In order to use this implementation you have to start bundle eclipse.smila.binarystorage.persistence.jpa in your launch configuration. Then set parameter implementationClass in file org.eclipse.smila.binarystorage.impl/BinaryStorageConfiguration.xml to org.eclipse.smila.binarystorage.persistence.jpa.JPABinaryPersistence. All other configuration options of file org.eclipse.smila.binarystorage.impl/BinaryStorageConfiguration.xml are ignored !
The other configuration needed is a typicall eclipseLink configuration property file. Therin you can specify settings for logging, database connection settings. For more information please refer to the eclipseLink documentation []. The configuration is located at configuration/org.eclipse.smila.binarystorage.persistence.jpa/persistence.properties.
# EclipseLink properties eclipselink.logging.level=INFO eclipselink.target-server=None eclipselink.target-database=org.eclipse.persistence.platform.database.DerbyPlatform eclipselink.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver eclipselink.jdbc.url=jdbc:derby:workspace/.metadata/.plugins/org.eclipse.smila.binarystorage.persistence.jpa/binarystorage;create=true eclipselink.jdbc.password=smila eclipselink.jdbc.user=smila eclipselink.ddl-generation=create-tables
After starting Smila for the first time, the DDL generation setting will print out some nasty warnings, complaining that it can't create some tables. These warnings are not critical. You can get rid of them by setting eclipselink.ddl-generation=none, but only after Smila was started at least once (and the tables were created).
At the moment it is necessary to import all packages containing JDBCDriver classes in org.eclipse.smila.connectivity.deltaindexing.jpa.impl. So for changing from derby to another database it is not sufficient to change the configuration in persistence.properties, you also have to add import package statementsv for the JDBC driver to use to your bundles manifest. This will hopefully be changed with the next release of eclipseLink.