Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "SMILA/Documentation/Binary Storage"

(Configuration example)
 
(4 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
[[Category:SMILA]]
 +
 
== Overview ==
 
== Overview ==
  
Line 39: Line 41:
 
* <tt>tempFileName</tt> - binary storage temporary file name
 
* <tt>tempFileName</tt> - binary storage temporary file name
 
* <tt>pathDepth</tt> - the depth of the folder structure (Note: If not set the default value is 2.)
 
* <tt>pathDepth</tt> - the depth of the folder structure (Note: If not set the default value is 2.)
 +
* <tt>path</tt> - you can optionally add a root folder to store the attachements at the given path.
  
 
== API ==
 
== API ==
Line 107: Line 110:
 
=== eclipse.smila.binarystorage.persistence.jpa ===
 
=== eclipse.smila.binarystorage.persistence.jpa ===
  
This implementation uses eclipseLink JPA to store binary objects in an apache derby database named <tt>BinaryStorage</tt>. The data is stored in table <tt>BINARY_OBJECTS</tt>:
+
This implementation uses eclipseLink JPA to store the binary objects in a database table named <tt>BINARY_OBJECTS</tt>.
 +
 
 +
The default configuration is using Apache Derby and the DB name is <tt>BinaryStorage</tt>.
  
 
{| class="wikitable" border="1" style="text-align:left"
 
{| class="wikitable" border="1" style="text-align:left"
Line 127: Line 132:
  
  
==== Configuration ====
+
In order to use this implementation you have to
 +
* set parameter <tt>implementationClass</tt> in file <tt>org.eclipse.smila.binarystorage.impl/BinaryStorageConfiguration.xml</tt> to <tt>org.eclipse.smila.binarystorage.persistence.jpa.JPABinaryPersistence</tt>
 +
* make sure that the bundle <tt>eclipse.smila.binarystorage.persistence.jpa</tt> starts in your launch configuration
  
In order to use this implementation set parameter <tt>implementationClass</tt> in file <tt>org.eclipse.smila.binarystorage.impl/BinaryStorageConfiguration.xml</tt>:
+
All other settings in <tt>BinaryStorageConfiguration.xml</tt> are ignored !
<source lang="text">
+
implementationClass="org.eclipse.smila.binarystorage.persistence.jpa.JPABinaryPersistence"
+
</source>
+
All other configuration options of file  <tt>org.eclipse.smila.binarystorage.impl/BinaryStorageConfiguration.xml</tt> are ignored !
+
  
The other configuration needed is a typicall eclipseLink configuration property file. Therin you can specify settings for logging, database connection settings. For more information please refer to the eclipseLink documentation [[http://wiki.eclipse.org/Using_EclipseLink_JPA_Extensions_(ELUG)#Using_EclipseLink_JPA_Extensions_for_JDBC]].
 
The configuration is located at <tt>configuration/org.eclipse.smila.binarystorage.persistence.jpa/persistence.properties</tt>.
 
 
<source lang="java">
 
# EclipseLink properties
 
eclipselink.logging.level=INFO
 
eclipselink.target-server=None
 
eclipselink.target-database=org.eclipse.persistence.platform.database.DerbyPlatform
 
eclipselink.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver
 
eclipselink.jdbc.url=jdbc:derby:workspace/.metadata/.plugins/org.eclipse.smila.binarystorage.persistence.jpa/binarystorage;create=true
 
eclipselink.jdbc.password=smila
 
eclipselink.jdbc.user=smila
 
eclipselink.ddl-generation=create-tables
 
</source>
 
  
After starting Smila for the first time, the DDL generation setting will print out some nasty warnings, complaining that it can't create some tables. These warnings are not critical. You can get rid of them by setting <tt>eclipselink.ddl-generation=none</tt>, but only after Smila was started at least once (and the tables were created).
 
  
==== Limitations ====
+
See [[SMILA/Documentation/General_JPA_Configuration_in_SMILA|General JPA Configuration in SMILA]] for a description on how to configure JPA based implementations.
At the moment it is necessary to import all packages containing JDBCDriver classes in <tt>org.eclipse.smila.connectivity.deltaindexing.jpa.impl</tt>. So for changing from derby to another database it is not sufficient to change the configuration in <tt>persistence.properties</tt>, you also have to add import package statementsv for the JDBC driver to use to your bundles manifest. This will hopefully be changed with the next release of eclipseLink.
+

Latest revision as of 10:05, 20 September 2011


Overview

Binary Storage Service provides an easy way to store / access binary data documents.

Client components access the Binary Storage Service for persisting binary data (attachments) into the binary storage. The binary data are identified by a unique key / identifier as a String data type.

Configuration

Currently, Binary Storage is able to run and store data into a hierarchical structure (uses deterministically calculation based on the hash id passed by the client component, like the ID passed from the Blackboard Service) and flat structure.

To configure the needed persistence storage structure, simply edit the /configuration/org.eclipse.smila.binarystorage/BinaryStorageConfiguration.xml file and change the value of implementationClass attribute to the one of the following available possibilities:

  • org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOHierarchicalManager
  • org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOFlatManager

In the next future there will be other available managers that provides the appropriate implementation for Binary Storage persistence structure.

The Binary Storage Configuration will be changed soon.

Configuration example

<BinaryStorageConfiguration xmlns="http://www.eclipse.org/smila/binarystorage" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:noNamespaceSchemaLocation="schemas/BinaryStorageConfiguration.xsd"
  name="default1"
  provider="file"
  mountPoint="default"
  implementationClass="org.eclipse.smila.binarystorage.internal.impl.persistence.filesystem.IOHierarchicalManager" 
  tempFileName="dummy.dat"
  pathDepth="2"
/>

Attributes explained:

  • name - configuration alias
  • implementationClass - binary storage implementation manager class
  • tempFileName - binary storage temporary file name
  • pathDepth - the depth of the folder structure (Note: If not set the default value is 2.)
  • path - you can optionally add a root folder to store the attachements at the given path.

API

void store(String id, InputStream stream);
void store(String id, byte[] blob)
byte[] fetchAsByte(String id)
InputStream fetchAsStream(String id)
void remove(String id)
int fetchSize(String id)

API Usage

BinaryStorageService binaryStorageService = ...; // Obtain the binary storage service
String attacmentIdKey = ...; // Unique ID
byte[] dataRecord = ... // Build record
 
// Store the binary record as byte array
binaryStorageService.store(attacmentIdKey, dataRecord);
 
// Fetch record as stream
final InputStream stream = binaryStorageService.fetchAsStream(attacmentIdKey);
 
// Define new ID and store new data from the first record's input stream
final String newAttacmentIdKey = attacmentIdKey + "0000";
binaryStorageService.store(newAttacmentIdKey, stream);
stream.close();
 
// Fetch the newest record as array of byte 
final byte[] recordByte = binaryStorageService.fetchAsByte(newAttacmentIdKey);
 
// The two records must have the same size
final int sizeInitial = _binaryStorageService.fetchSize(attacmentIdKey);
final int sizeFinal = binaryStorageService.fetchSize(newAttacmentIdKey);
 
// Remove the two records
binaryStorageService.remove(attacmentIdKey);
binaryStorageService.remove(newAttacmentIdKey);

Implementations

eclipse.smila.binarystorage.persistence.io

Both I/O implementations use the id in the path name of the persisted file. As a consequence

  • the characters [;/\:] are not allowed in the id
  • too long ids will exceed the platform specific supported max. path length. this will result in a not so obvious error message, that saving failed.

ATM neither limitation this is a problem because the id is a hash made from letters and numbers.

IOHierarchicalManager

The calculation of the path is simply to split the first part of the id into substrings of length 2 and make thees parent folders having the name of the id.

Care must be taken that the folder depth is not too high, because this results in a lot of folders quenching the performance of the file system.

IOFlatManager

If u only have a few files or ur underlying file system is very good inhandling lots of files per folder then go with this manager.


eclipse.smila.binarystorage.persistence.efs

This implementation is based on EFS.


eclipse.smila.binarystorage.persistence.jpa

This implementation uses eclipseLink JPA to store the binary objects in a database table named BINARY_OBJECTS.

The default configuration is using Apache Derby and the DB name is BinaryStorage.

BINARY_OBJECTS
Column Type Description
ID VARCHAR the Id of the binary object
BIN_OBJECT BLOB the binary object


In order to use this implementation you have to

  • set parameter implementationClass in file org.eclipse.smila.binarystorage.impl/BinaryStorageConfiguration.xml to org.eclipse.smila.binarystorage.persistence.jpa.JPABinaryPersistence
  • make sure that the bundle eclipse.smila.binarystorage.persistence.jpa starts in your launch configuration

All other settings in BinaryStorageConfiguration.xml are ignored !


See General JPA Configuration in SMILA for a description on how to configure JPA based implementations.

Copyright © Eclipse Foundation, Inc. All Rights Reserved.