Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Orion/Server API/Transfer API"

(Export)
m (SFTP export to another server)
 
(37 intermediate revisions by 5 users not shown)
Line 1: Line 1:
'''Under Construction: This API is not yet implemented'''
 
 
 
The transfer API is used for batch import and export of directory trees to/from an Orion workspace.
 
The transfer API is used for batch import and export of directory trees to/from an Orion workspace.
  
Line 6: Line 4:
  
 
== Starting a file upload ==
 
== Starting a file upload ==
 +
 +
'''** NOTE: The Orion Java server currently returns an invalid Location value in its response, which makes the approach of pairing this call with subsequent calls to the "Uploading a file" endpoint not work.  The Orion client instead uses the "Importing a file via a single POST call" endpoint to perform its file uploads.'''
  
 
{{Orion/ServerAPI
 
{{Orion/ServerAPI
 
| method = POST
 
| method = POST
| overview = An upload is initiated via a POST request. The request URL indicates where the file should be located once the upload is complete. The request must indicate the total size of the file the server should expect.
+
| overview = A resumable chunked upload is initiated via a POST request. The request URL indicates where the file should be located once the upload is complete. The request must indicate the total size of the file the server should expect.
| reqhead = /xfer/MyProj/SomeFolder
+
| reqhead = /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}
 
  Orion-Version: 1.0
 
  Orion-Version: 1.0
 
  Slug: MyTitle
 
  Slug: MyTitle
 
  X-Xfer-Content-Length: 901024
 
  X-Xfer-Content-Length: 901024
 +
X-Xfer-Options: raw
 
| resphead = 200 OK
 
| resphead = 200 OK
 
  Location: /xfer/import/23A43
 
  Location: /xfer/import/23A43
| explain = The Location header in the response indicates the URL where chunks of the file should be uploaded. Once the server has received all chunks, the import into the workspace is completed. An upload that is not completed in a timely fashion from the initial POST is discarded by the server.
+
| explain = The Location header in the response indicates the URL where chunks of the file should be uploaded. Once the server has received all chunks, the import into the workspace is completed. An upload that is not completed in a timely fashion from the initial POST is discarded by the server. The X-Xfer-Options header specifies what the server should do with the file when the upload completes. By default the server assumes the file is an archive and will attempt to extract it into the destination location. When importing a single file, the "raw" option can be used to indicate that the server should simply place the uploaded file at the specified location.
 
}}
 
}}
  
Line 33: Line 34:
 
| resphead = 308 Resume Incomplete
 
| resphead = 308 Resume Incomplete
 
  Range: bytes 0-30000
 
  Range: bytes 0-30000
| explain = A response of 308 indicates that the server has not yet received all required bytes, and the Range response header indicates the bytes known to the server so far. When the server has received all bytes, a 201 Created response is returned, with a Location header specifying the location of the newly added file or directory.
+
| explain = A response of 308 indicates that the server has not yet received all required bytes, and the Range response header indicates the bytes known to the server so far. When the server has received all bytes, a 201 Created response is returned, with a Location header specifying the location of the newly added file or directory.  
 
}}
 
}}
  
Line 43: Line 44:
 
| method = POST
 
| method = POST
 
| overview = An upload is initiated via a POST request. The request URL indicates where the file should be located once the upload is complete. The request must indicate the total size of the file the server should expect.
 
| overview = An upload is initiated via a POST request. The request URL indicates where the file should be located once the upload is complete. The request must indicate the total size of the file the server should expect.
| reqhead = /xfer/MyProj/SomeFolder
+
| reqhead = /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}
 
  Orion-Version: 1.0
 
  Orion-Version: 1.0
 
  X-Xfer-Content-Length: 901024
 
  X-Xfer-Content-Length: 901024
X-Xfer-Options: unzip
 
 
| resphead = 200 OK
 
| resphead = 200 OK
 
  Location: /xfer/import/23A43
 
  Location: /xfer/import/23A43
Line 52: Line 52:
 
}}
 
}}
  
 +
== Importing from SFTP==
  
 +
SFTP import is a server to server operation for pulling content from an SFTP server into an Orion workspace.
 +
 +
{{Orion/ServerAPI
 +
| method = POST
 +
| overview = An import is initiated via a POST request. The request URL indicates where the file should be located once the upload is complete. The X-Xfer-Options header indicates the type of import, and the request body provides details required to complete the import.
 +
| reqhead = /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}
 +
Orion-Version: 1.0
 +
Content-Type: application/json
 +
X-Xfer-Options: sftp
 +
| reqbody = {
 +
  "UserName" : "nanush",
 +
  "Host" : "example.com",
 +
  "Port" : "22",
 +
  "Path" : "/nanush/MyProj",
 +
  "Passphrase": "secretpassword"
 +
}
 +
| resphead = 201 Created
 +
Location: /task/dfsfSDF7S_F7
 +
| explain = The Location header in the response indicates the URL of a progress resource that can be used to monitor the state of the import operation.<br /><br />By default, this import will overwrite any files in your workspace with the same name. An X-Xfer-Options value of "sftp,no-overwrite" indicates that the operation MUST fail if the resource to be created already exists. A 412 return code is used to indicate that the "no-overwrite" option was specified and the destination resource already exists. An X-Xfer-Options value of "sftp,overwrite-older" will overwrite any files in your workspace with a timestamp older than the file on the source server.<br /><br />The Port field is optional; if not provided port 22 is used.
 +
}}
 +
 +
== Importing a file via a single POST call ==
 +
 +
{{Orion/ServerAPI
 +
| method = POST
 +
| overview = Although the resumable chunk-based upload approach is recommended for large files, you can also import a file with a single POST call. Simply omit the X-Xfer-Content-Length header, and include the file to be uploaded as the request body.
 +
| reqhead = /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}
 +
Orion-Version: 1.0
 +
Content-Length: 901024
 +
Content-Type: application/zip
 +
| reqbody = This is the file contents
 +
| resphead = 201 OK
 +
Location: /file/user-OrionContent/MyProj/SomeFolder
 +
| explain = If the POST method body contains the entire content, then 201 response is returned and the import is complete. If the entire content is not received, then a 308 response is returned along with the range received by the server. The client can then resume or complete the upload via a subsequent PUT operation with appropriate range. The file is only imported into the workspace once the entire content is received.
 +
}}
 +
 +
== Import a file from a remote URL ==
 +
 +
{{Orion/ServerAPI
 +
| method = POST
 +
| overview = You can also import a file located at an arbitrary remote URL. This example imports an HTML file from http://example.com/files.html, and imports it into SomeFolder on the Orion server.
 +
| reqhead = /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}?source=http%3A%2F%2Fexample.com%2Findex.html
 +
Orion-Version: 1.0
 +
X-Xfer-Options: raw
 +
| resphead = 201 OK
 +
Location: /file/user-OrionContent/MyProj/SomeFolder
 +
| explain = The '''X-Xfer-Options''' header specifies what the server should do with the file when the upload completes. By default the server assumes the file is an archive and will attempt to extract it into the destination location. When importing a single file, the "raw" option can be used to indicate that the server should simply place the uploaded file at the specified location.
 +
}}
  
 
= Export =
 
= Export =
  
* GET request /xfer/export/'''pathToExport'''.zip
+
== Export to client ==
* Request Content-Type indicates archive format (default=application/zip)
+
 
 +
{{Orion/ServerAPI
 +
| method = GET
 +
| overview = Exporting files and directories to the client is performed via a GET
 +
| reqhead = /xfer/export/{:workspace_id}/{:project_dir}/{:folder_path}.zip
 +
Orion-Version: 1.0
 +
Content-Type: application/zip
 +
| resphead = 201 OK
 +
Content-Length: 901024
 +
Content-Type: application/zip
 +
| respbody = File contents go here
 +
| explain = The final segment of the request must be the name of an existing workspace folder with ".zip" appended (for example, to export the contents of folder /a/b/c your :folder_path must be /a/b/c.zip). The body of the response will be the exported files/directories in the requested format. If no Content-Type is specified in the request, then the default response type is application/zip.
 +
}}
 +
 
 +
== SFTP export to another server ==
 +
SFTP export is a server-to-server operation for pushing content from an Orion workspace to an SFTP server.
 +
 
 +
{{Orion/ServerAPI
 +
| method = POST
 +
| overview = A server to server export is initiated via a POST request. The request URL indicates the file or directory to export. The X-Xfer-Options header indicates the type of export, and the request body provides details required to complete the export.
 +
| reqhead = /xfer/export/{:workspace_id}/{:project_dir}/{:folder_path}/
 +
Orion-Version: 1.0
 +
Content-Type: application/json
 +
X-Xfer-Options: sftp
 +
| reqbody = {
 +
  "UserName" : "nanush",
 +
  "Host" : "example.com",
 +
  "Port": "22",
 +
  "Path" : "/nanush/MyProj",
 +
  "Passphrase": "secretpassword"
 +
}
 +
| resphead = 201 Created
 +
Location: /task/dfsfSDF7S_F7
 +
| explain = The Location header in the response indicates the URL of a progress resource that can be used to monitor the state of the export operation.<br /><br />By default this export will overwrite any files in the destination server with the same name. An X-Xfer-Options value of "sftp,no-overwrite" indicates that the operation MUST fail if the resource to be exported already exists. A 412 return code is used to indicate that the "no-overwrite" option was specified and the destination resource already exists. An X-Xfer-Options value of "sftp,overwrite-older" will overwrite any files in the remote server with a timestamp older than the file in the source workspace.<br /><br />The Port field is optional; if not provided port 22 will be used.
 +
}}
 +
 
 +
[[Category:Orion/API|Transfer API]]
 +
[[Category:Orion/Server API|Transfer]]

Latest revision as of 16:08, 24 May 2017

The transfer API is used for batch import and export of directory trees to/from an Orion workspace.

Import

Starting a file upload

** NOTE: The Orion Java server currently returns an invalid Location value in its response, which makes the approach of pairing this call with subsequent calls to the "Uploading a file" endpoint not work. The Orion client instead uses the "Importing a file via a single POST call" endpoint to perform its file uploads.


Overview
A resumable chunked upload is initiated via a POST request. The request URL indicates where the file should be located once the upload is complete. The request must indicate the total size of the file the server should expect.
HTTP Method
POST
Example Request
POST /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}
Orion-Version: 1.0
Slug: MyTitle
X-Xfer-Content-Length: 901024
X-Xfer-Options: raw

  
Example Response
HTTP/1.1 200 OK
Location: /xfer/import/23A43


Detailed Explanation
The Location header in the response indicates the URL where chunks of the file should be uploaded. Once the server has received all chunks, the import into the workspace is completed. An upload that is not completed in a timely fashion from the initial POST is discarded by the server. The X-Xfer-Options header specifies what the server should do with the file when the upload completes. By default the server assumes the file is an archive and will attempt to extract it into the destination location. When importing a single file, the "raw" option can be used to indicate that the server should simply place the uploaded file at the specified location.


Uploading a file

After initiating the transfer, uploads are performed as many times as required using PUT actions:


Overview
Uploads a chunk of a file to the server.
HTTP Method
PUT
Example Request
PUT /xfer/import/23A43
Orion-Version: 1.0
Content-Length: 32768
Content-Type: application/zip
Content-Range: bytes 0-32767/901024

  
Example Response
HTTP/1.1 308 Resume Incomplete
Range: bytes 0-30000


Detailed Explanation
A response of 308 indicates that the server has not yet received all required bytes, and the Range response header indicates the bytes known to the server so far. When the server has received all bytes, a 201 Created response is returned, with a Location header specifying the location of the newly added file or directory.


Importing a zip

The chunked file upload service can also be used to import an archive file that will be expanded upon completion of import. This allows a large number of files to be imported into a project at once.


Overview
An upload is initiated via a POST request. The request URL indicates where the file should be located once the upload is complete. The request must indicate the total size of the file the server should expect.
HTTP Method
POST
Example Request
POST /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}
Orion-Version: 1.0
X-Xfer-Content-Length: 901024

  
Example Response
HTTP/1.1 200 OK
Location: /xfer/import/23A43


Detailed Explanation
The Location header in the response indicates the URL where chunks of the file should be uploaded. Once the server has received all chunks, the file is unzipped into the destination location on the server. An upload that is not completed in a timely fashion from the initial POST is discarded by the server.


Importing from SFTP

SFTP import is a server to server operation for pulling content from an SFTP server into an Orion workspace.


Overview
An import is initiated via a POST request. The request URL indicates where the file should be located once the upload is complete. The X-Xfer-Options header indicates the type of import, and the request body provides details required to complete the import.
HTTP Method
POST
Example Request
POST /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}
Orion-Version: 1.0
Content-Type: application/json
X-Xfer-Options: sftp

{
 "UserName" : "nanush",
 "Host" : "example.com",
 "Port" : "22",
 "Path" : "/nanush/MyProj",
 "Passphrase": "secretpassword"
}  
Example Response
HTTP/1.1 201 Created
Location: /task/dfsfSDF7S_F7


Detailed Explanation
The Location header in the response indicates the URL of a progress resource that can be used to monitor the state of the import operation.

By default, this import will overwrite any files in your workspace with the same name. An X-Xfer-Options value of "sftp,no-overwrite" indicates that the operation MUST fail if the resource to be created already exists. A 412 return code is used to indicate that the "no-overwrite" option was specified and the destination resource already exists. An X-Xfer-Options value of "sftp,overwrite-older" will overwrite any files in your workspace with a timestamp older than the file on the source server.

The Port field is optional; if not provided port 22 is used.


Importing a file via a single POST call

Overview
Although the resumable chunk-based upload approach is recommended for large files, you can also import a file with a single POST call. Simply omit the X-Xfer-Content-Length header, and include the file to be uploaded as the request body.
HTTP Method
POST
Example Request
POST /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}
Orion-Version: 1.0
Content-Length: 901024
Content-Type: application/zip

This is the file contents  
Example Response
HTTP/1.1 201 OK
Location: /file/user-OrionContent/MyProj/SomeFolder


Detailed Explanation
If the POST method body contains the entire content, then 201 response is returned and the import is complete. If the entire content is not received, then a 308 response is returned along with the range received by the server. The client can then resume or complete the upload via a subsequent PUT operation with appropriate range. The file is only imported into the workspace once the entire content is received.


Import a file from a remote URL

Overview
You can also import a file located at an arbitrary remote URL. This example imports an HTML file from http://example.com/files.html, and imports it into SomeFolder on the Orion server.
HTTP Method
POST
Example Request
POST /xfer/import/{:workspace_id}/{:project_dir}/{:folder_path}?source=http%3A%2F%2Fexample.com%2Findex.html
Orion-Version: 1.0
X-Xfer-Options: raw

  
Example Response
HTTP/1.1 201 OK
Location: /file/user-OrionContent/MyProj/SomeFolder


Detailed Explanation
The X-Xfer-Options header specifies what the server should do with the file when the upload completes. By default the server assumes the file is an archive and will attempt to extract it into the destination location. When importing a single file, the "raw" option can be used to indicate that the server should simply place the uploaded file at the specified location.


Export

Export to client

Overview
Exporting files and directories to the client is performed via a GET
HTTP Method
GET
Example Request
GET /xfer/export/{:workspace_id}/{:project_dir}/{:folder_path}.zip
Orion-Version: 1.0
Content-Type: application/zip

  
Example Response
HTTP/1.1 201 OK
Content-Length: 901024
Content-Type: application/zip

File contents go here
Detailed Explanation
The final segment of the request must be the name of an existing workspace folder with ".zip" appended (for example, to export the contents of folder /a/b/c your :folder_path must be /a/b/c.zip). The body of the response will be the exported files/directories in the requested format. If no Content-Type is specified in the request, then the default response type is application/zip.


SFTP export to another server

SFTP export is a server-to-server operation for pushing content from an Orion workspace to an SFTP server.


Overview
A server to server export is initiated via a POST request. The request URL indicates the file or directory to export. The X-Xfer-Options header indicates the type of export, and the request body provides details required to complete the export.
HTTP Method
POST
Example Request
POST /xfer/export/{:workspace_id}/{:project_dir}/{:folder_path}/
Orion-Version: 1.0
Content-Type: application/json
X-Xfer-Options: sftp

{
 "UserName" : "nanush",
 "Host" : "example.com",
 "Port": "22",
 "Path" : "/nanush/MyProj",
 "Passphrase": "secretpassword"
}  
Example Response
HTTP/1.1 201 Created
Location: /task/dfsfSDF7S_F7


Detailed Explanation
The Location header in the response indicates the URL of a progress resource that can be used to monitor the state of the export operation.

By default this export will overwrite any files in the destination server with the same name. An X-Xfer-Options value of "sftp,no-overwrite" indicates that the operation MUST fail if the resource to be exported already exists. A 412 return code is used to indicate that the "no-overwrite" option was specified and the destination resource already exists. An X-Xfer-Options value of "sftp,overwrite-older" will overwrite any files in the remote server with a timestamp older than the file in the source workspace.

The Port field is optional; if not provided port 22 will be used.

Back to the top