Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Flux/Prototype
This is a brief description of the technical details of the Flux prototype implementation.
async messaging - the foundation
The back-bone of the Flux prototype implementation is an asynchronous messaging channel. The basic idea behind this architecture is that all communication in Flux is happening via asynchronous messages. There is no RESTful API, a dedicated server that can be called, or something like that. Instead every component in Flux is connected to this messaging channel and can send and receive messages over this wire. Everything else is implemented on top of this.
The prototype implementation of the asynchronous messaging channel is based on websockets. Since this is the only way to connect to Flux, every participant can open a websocket connection to the messaging system and send and receive messages from there on. The websocket messaging server is implemented as a node.js application written in JavaScript:
Messages in the Flux prototype are in JSON notation. They can either be broadcasted to everybody in the system or send directly to a certain recipient. This is configured for each socket that is opened:
Different users are differentiated via different spaces using the room feature of the socket.io library. Therefore after connecting to the websocket server, every participant has to send a message "connectToChannel" to let that websocket connection participate in the channel for that particular user. As a result, if messages are broadcasted, they are not broadcasted "to the world", but within the channel of that user only. This is good to security as well as for keeping the traffic per socket down to the messages that are interesting for that user only.
As an additional step, the connect to the websocket itself should be secured, so that only authenticated users can successfully connect to the websocket. In addition to that the reaction to the "connectToChannel" message can be diversified by checking whether the user is allowed to enter a certain channel. By default this would be the same (the authenticated user connects to his own channel), but it could also include something like an invitation mechanism, so that users can invite and allow other users to take a look at their resources in Flux.
file sync
the file syncing in the Flux prototype is implemented on top of the asynchronous messaging. The underlying design decision is that there is no single master copy of the files/folders/projects. Instead, each participant in the Flux system (all the components that are connected to the messaging channel) can serve as a "repository" for individual files, complete projects, everything from a user, or everything in the Flux universe. To sync the content of files and projects, each participant is responsible for listening to the file sync messages, replying to file sync request messages, and to broadcast messages about changes. That creates a network of participants that sync those files and changes among each other (instead of relying on a master server that stores and keeps track of everything).
The syncing mechanism uses the following messages at the moment:
- projectConnected: A new project is connected to Flux. This makes other participants aware of a new project being connected to Flux.
- projectDisconnected: A project is disconnected from Flux. Since this means that a single participant has disconnected the project from Flux, is doesn't mean that all participants have disconnected this project.
- resourceCreated: A new resource got created (as part of a project). This can be a file or a folder.
- resourceChanged: A resource got changed (like a save event). This can be a file, a folder, or a project
- resourceDeleted: A resource got deleted (can be a folder or a file)
- resourceStored: A resource got stored. This means that a resource change as been stored persistently by a participant in the system. This is usually a reaction to a resourceChanged event in the case that the resource change got safely stored somewhere.
- getProjectsRequest: A request to send the requester back information about the projects that are connected to Flux.
- getProjectsResponse: The response, containing the list of projects that are connected to Flux
- getProjectRequest: A request to send back information about the content of an individual project (like the list of contained resources)
- getProjectResponse: The response, containing metadata and a list of contained resources.
- getResourceRequest: A request for the content of an individual resource (e.g. the content of a file)
- getResourceResponse: The response, containing the content of the resource (e.g. the content of a file)
At the moment the file sync protocol doesn't deal with conflicts and/or parallel file changes. This has to be added to the mechanism while moving the project forward.