Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
PTP/designs/hpc user workflow
HPC User Workflow
PTP is designed to interact with a variety of computer systems in order to aid the development of parallel and scientific applications. Many of these systems are extremely large, comprising many tens or hundreds of thousands of cores. In addition, the applications that run on these systems can also be very large, not just in terms of the number of lines of code, but also in their memory usage and the number of computational tasks required to deliver adequate performance.
Providing a development environment that allows developers to maximize their productivity in these environments is a challenging task. Not only do the applications need to scale, but the tools that are required to monitor, debug, and analyze these application must also scale.
The following document outlines the design elements of PTP that are key to ensuring that it will continue provide a useful and productive development platform, regardless of the system or application scaling requirements.
Typical HPC User
The typical HPC User needs both tools to develop applications but also tools to manage builds, compilation, runs and system connections. This project aims to refocus the usability of Eclipse PTP to fit the workflow of the typical HPC user who might have multiple projects and a need access to multiple systems to build and run their projects.
Examples of workflows
Here are some sample case scenarios we're are trying to incorporate into a solution
- A user who takes an already community built application and wishes to use it to get scientific results on one or several systems the user has access to.
- A user may wish to develop a new application on a test bed then deploy on other test/production HPC systems.
- A user is responsible for several projects as a staff scientists and needs to deploy on one or several systems
- A user wishes to add to a codebase that alread exists on a particular system (and may/may not want to migrate that code to another system)
- A user migrating code from System A to System B may wish to replicate the environment used in the development machine. (same compiler, version, library, if available, via modules)
- A System Administrator may wish to export information about the systems available to users for parsing by Eclipse so that settings are transparent to the user.
- A user may wish to explore in a single view which shows his/her accessible systems, availability and their load.
- A user may wish to replicate/launch several runs from one application and monitor those runs. (Wyatt working on this in ETFw)
Ideas for usability
When syncing files that are either very large or very numerous, before syncing a user should be given the choice to: know which files have changed from when it last synced, and whether or not any other files should be synced. Perhaps a popup that says "we noticed these files have changed and these files were added, would you like to sync? (check all)(continue with current checked)(preview changes on this File A)
Adding paths to the project isn't straight forward. (especially in the case where scientists need to append data directories)
Adding the capacity for dynamic Toolchains where a project is created and a tool chain can be chosen later and or switched.