Hawk and Sol Usage Policy

Hawk, Lehigh's newest HPC cluster funded by NSF Campus Cyberinfrastructure grant provides 15.347M core-hours or service units (SUs) of computing per year that are distributed as follows

• PI Team (50%): 7,673,760 SUs
• OSG (20%): 3,069,504 SUs
• Lehigh HPC (25%): 3,836,880 SUs 
• Research: 3,453,192 SUs
• Startup: 383,688 SUs
• Provost’s Discretionary Fund (DF): 767,376 SUs

Lehigh’s investment in Sol (Hotel SUs) provides 1.4M SUs will be added to the above pool of Hawk resources to top off Lehigh HPC to 4.5M SUs and the Provost’s DF to 1M. The remaining will be combined with HPC’s share from the PI Team allocation to create a new Director’s Discretionary Fund.

Director’s Discretionary Fund (DDF)

As described in the proposal,  DDF will be used for supporting the two REU programs, at least two courses each Fall and Spring, and one HPC Workshop in the summer.  The added SUs from SOL will be used for the following additional purposes - more courses per semester including capstone projects, summer research programs and workshops, and, trial allocations to new research groups for exploring Hawk’s capabilities. Unused SUs will be provided in order of precedence to Lehigh StartUp, Provost DF, and Lehigh Research.

Provost's Discretionary Fund (PDF)

These are allocations to be distributed at the discretion of Lehigh's Provost.

Research Computing Allocation Review Committee

Writing an Allocation Proposal

Research Computing providing a webinar on writing an allocation proposal on Wednesday, Dec 1 at 2PM. 

Slides

Effective October 1, 2020,  there is no charge to use Lehigh's HPC resources - Sol and Hawk. Instead, Faculty are required to submit an allocation proposal that will be reviewed by a faculty driven Research Computing Allocation Review Committee (RCARC). The RCARC has representation from all five Colleges to ensure each College is involved in the decision making process to distribute HPC resources.

Accounts

Lehigh Faculty, research staff and students can request an account on Sol/Hawk. All account requests by staff and students need to be sponsored by a Lehigh Faculty and be associated with an active allocation. A Lehigh faculty with no active allocation, can request an account for themselves and members of their group while requesting an allocation as described below. Accounts that do not have an active allocation for 6 months will be disabled. 

Allocations

An allocation is a block of computing time measured in core-hours or service units or SUs (number of processing cores requested times the amount of wall-clock time used in hours). There is no charge to using Sol/Hawk  resources. However, users need to have an allocation to charge simulation time. There are different types of allocation depending on amount of SUs and storage.

Eligibility

Only Full Time Lehigh faculty are eligible to be a PI on an allocation. Other users such as research staff including postdocs, graduate, and undergraduate students need to be associated with a Lehigh Faculty allocation. A user can be a member of multiple allocations, either the same faculty or different.

Trial

Trial Allocations are geared towards first time HPC faculty who want to learn more about the capabilities of Sol/Hawk before requesting a startup or research allocation. Each allocation request will provide a maximum 10K SUs with a 6 month validity. 

Requirements: Faculty requesting a trial allocation should provide a short description of the work they wish to accomplish.

Request Window: Trial Allocations can be requested any time during the year and should be approved within 2-3 Business days.

Allocation Review: Approving trial allocations will be at the discretion of the Manager of Research Computing from the Director’s Discretionary Fund. 

Renewal: Trial Allocations can be renewed on an annual basis after the 6 month validity. Renewals will be approved by at least two RCARC members. Approvals could take 3-4 weeks and may need to provide additional information at the discretion of RCARC members.

Eligibility Restriction: A Faculty who already has a startup or research allocation is not eligible for Trial Allocations. Trial Allocations are not a short term allocation block while waiting for the next allocation review cycle.  

Apply here

StartUp

Startup Allocations are small allocation requests (<25K). A faculty can be a PI or collaborator on only two active Startup Allocation at any time. 

Requirements: Faculty requesting a StartUp Allocation should provide a short abstract (min 500 words, 1 page max) with the description of the work they wish to accomplish and the amount of computing time requested.

Request Window: StartUp Allocations can be requested any time during the year and will be approved within 3-4 weeks of submission (completed).

Allocation Review: StartUp Allocations will be reviewed by the RCARC. Any RCARC member can review and approve StartUp requests. 

Renewal: StartUp Allocations can be renewed annually by submitting a progress report to the RCARC in addition to the requirements mentioned above. Renewal requests would need to be approved by at least 2 RCARC members, could take up to 4 weeks, and require additional information at the discretion of RCARC members.

Restriction: Due to the ease of obtaining StartUp Allocation, the following restriction would apply:

• Startup allocations that are expended within 6 months of the start date will be denied. PIs of such accounts will need to request a Research Allocation as described below.
• Any user can be associated with only 2 startup allocations at any given time.
  • The RCARC may request additional information if the StartUp Allocation request contains users who are already associated with a StartUp Allocation.
  • Requests to add existing users who have a StartUp Allocation after approval will automatically trigger a review by RCARC.

Apply here

Education

Developing the next generation workforce is a cornerstone of NSF campus research cyberinfrastructure grants. Use of Hawk for education - course work and workshops, is strongly encouraged.

The Manager of Research Computing will oversee allocation of computing time for education, outreach and training purposes. 

Courses: A minimum of 500 SU/student is recommended. The maximum available SU per course is 50K SUs and each course will receive a 1TB Ceph space. These allocations will expire at the end of the semester while the Ceph space will be available to the Instructor only for a period of 1 year to encourage them to teach the course and reuse the course material annually. 

Capstone projects: Capstone projects are considered to be a single course and are eligible for only one allocation. If the Instructor requires more than 50K SUs or requires multiple allocations for different groups of users, then a startup or research allocation should be requested. Such requests will be reviewed by the RCARC who may request for additional information.

Workshops/Summer Programs: Faculty, Departments, Institutes or Colleges will be eligible for allocations (max 10K) to support workshops and summer programs centered on scientific computing. PIs of summer programs such as Research Experience for Undergraduates can add students to their own startup or research allocation or apply for one (recommended for projects that will continue beyond the summer).

At the end of the semester or program, the Instructor or Program PI will need to provide a progress report.  Future education allocations will not be prioritized or even denied if a progress report is not completed.

Please take note that there will be no special arrangements i.e. reservations created for course allocations. These allocations are ideal for project based course work rather than homework assignments. It is the instructors responsibility to raise awareness that Sol and Hawk are research equipments and the purpose of using them for the course work is to educate students on the use of research equipment i.e. a shared, schedulable resource. If you need an on-demand resource, please use LUApps or Cloud Computing resources.

We will however entertain reasonable requests for reservations or dedicated access to resources during the final week of course or for the duration of a workshop (max 1 week).

Please fill out a request and provide the following information.

1. Course Number, Workshop Title, Summer Research Program Name
2. Brief description on how resources will be used.
3. Approximate number of users in the program.
   1. If number of users is greater than 25, consider discussing with A. Pacheco the appropriateness of Lehigh Resources vs XSEDE (or cloud computing).
4. List of applications to be used.

Research

Research Allocations are allocation requests larger than StartUp (>25K) with a maximum 300K allocation per Faculty.  A faculty can have multiple research allocations corresponding to multiple research projects. The total active SUs across all such projects including startup allocations should not exceed 300K at any time.
For example : John Doe has 4 existing projects with 30K, 40K, 80K, and 100K respectively for a total allocation of 250K SUs. John can be approved for an allocation provided the request is 50K SUs or less. Jane Doe who has similar projects but the smallest one expires at the end of the allocation cycle in 30 days can request a renewal with a maximum of 80K SUs. Jane’s total allocation until the end of the current cycle will be 250K and a total of 300K when the next cycle begins if the request is approved.

Active allocation is defined as an allocation that has not reached its end date. An expended allocation is active before its end date, it just cannot be used to charge computing time.

Requirement: A faculty requesting  a research allocation needs to submit a detailed research proposal. The proposal should contain the following -  

1. Abstract of the project (max 0.5 page)

2. Description of the research project and what you wish to accomplish in a year (max 2 pages). 

3. Description of how Sol/Hawk will be used or Benchmarks of simulations to be carried out (max 4 pages)

1. How many simulations need to run and application used?

2. How does the application scale?

3. Approximate simulation length and amount of data generated/stored per simulation

4. Total Allocations including Storage requested - should be justified by a-c above. 

4. Other Information - required for renewals or prior Sol users.

   1. e.g. NSF award number, title, list of PIs, Amount awarded, Start-End Date. (Optional - 25% of the project involves use of computational resource)

   2. Have you budgeted computing time in your grant?

1. Publications, presentations, invited talks based on results generated by prior allocation.

2. Grants submitted or awarded. If a small portion is computational, please provide a %. Include allocations awarded on national resources.

3. Students graduated since previous award with thesis/dissertation title

Request Window: Research Allocations can be submitted during a one-month window every six months. Tentatively, there will be two allocation cycles with start dates of Jan 1 and Jul 1. Allocation window would be open two months prior to the start dates i.e. Nov 1 and May 1 respectively. (overlaps with XSEDE cycle 12/15-1/15 and 10/15-11/15).

Allocation Review: RCARC will review and approve allocations following the deadline but prior to the start of the allocation cycle i.e. in Dec and June every year. Please note that the RCARC is not reviewing your research project for its content or quality. The RCARC is only evaluating whether your request for resources match your proposed work. You cannot request 100K SUs when you need to run 10 jobs for 200 hours on 10 CPUs i.e. 2000 SUs each, 20K max. Depending on the total amount of SUs requested and the amount available, RCARC may approve your request with a reduced amount.

Renewal: Research allocations can be renewed annually. Renewals require a progress report of prior award and other information described in item 4 of requirements above.
Renewals that do not include a progress report will be reviewed and awarded but allocations will be available only after submission of the progress report. If a progress report is not provided within a month of the start date i.e. Feb 1 or Aug 1 depending on the cycle, then the allocation award will be withdrawn and all future requests will be declined in the absence of a progress report. E.g. John Doe requested a renewal of research allocation of 200K SUs that did not include a progress report. The RCARC will review the request and award an allocation subject to submission and review of the progress report within a month, say Jan 31. John and his collaborators do not have access to the 200K SUs until the report is submitted. If by Jan 31, the report is not submitted, then the 200K SU is returned to the RCARC, who can distribute at their discretion. On June 30, John’s other allocation, 100K, expires and he submits his renewal package by the May 31 deadline. If this renewal request does not contain the progress report for both the allocations, his renewal request will not be reviewed.

Apply here

Example Proposals (EES, Math)

Storage

Each HPC PI will be provided with a 1TB storage space that will provide home directory space of all accounts sponsored by that PI. PIs who have a large group can request a large space for home storage such that 1TB is provided for every 5 active users. It is the PI's responsibility to move inactive user's data to a backup storage solution and make room for active users. PI's who do not have an active allocation and have not applied for an allocation within 6 months of the expiry of the previous award will have the 1TB storage withdrawn with all data deleted with no backups. This space will be provided from the 40TB DDF. PIs who unnecessarily add users to the allocation to request more storage space will be restricted to 1TB space in subsequent years unless approved by RCARC.

The 1TB space provided here is only valid while you have an active allocation. If you do not have an active allocation, your data will be deleted freeing space that can be provisioned to other PI's with active allocations. Purchased Ceph space has the 5 year life and does not require an active allocation.

RCARC will allocate up to 40TB (20TB each allocation cycle) as part of the research allocation request.

• HPC PIs can combine their computing and storage requests in a single proposal. Storage approved by RCARC will be provided as Ceph project spaces at /share/ceph/{project_name}.
• Non HPC PIs, the space will appear as an additional folder when you mount R Drive.

This space cannot be combined with the free 1TB per HPC PI, R Drive or Ceph spaces purchased at $375/TB for 5 years. HPC does not backup Ceph (R Drive is a Ceph project)  nor do we provide backup services. Faculty are responsible for backing up their data and archiving it for later use.

Storage allocation needs to be renewed annually. Storage projects that are not renewed, either declined or did not renew, will be deleted one month after the expiry of the allocation so that another PI who is allocated space can utilize it. 

Requirement: A faculty requesting  a storage allocation needs to submit a detailed research proposal. The proposal should contain the following -  

1. Abstract of the project (max 0.5 page)
2. Description of the research project and what you wish to accomplish in a year (max 2 pages). 
3. Describe how you plan to use the storage allocation and why other storage resources (list resources considered) are not a good option. (max 1 page)
4. Describe how you plan to backup data stored in this allocation. (½ page)
5. If your project is not renewed next year, how do you plan to  move/save the data. (½ page) Note: if the project is not renewed, you will have a month to transfer data before the Ceph space is deleted and made available to other PIs who have a successful storage allocation. 
6. Other Information - required for renewals or prior Sol users.
   1. e.g. NSF award number, title, list of PIs, Amount awarded, Start-End Date. (Optional - 25% of the project involves use of computational resource)
   2. Have you budgeted storage in your grant?
   3. Publications, presentations, invited talks based on results generated by prior allocation.
   4.  Grants submitted or awarded. 
   5. Students graduated since previous award with thesis/dissertation title

Request Window: Storage Allocations can be submitted during a one-month window every six months. Tentatively, there will be two allocation cycles with start dates of Jan 1 and Jul 1. Allocation window would be open two months prior to the start dates i.e. Nov 1 and May 1 respectively. 

Allocation Review: RCARC will review and approve allocations following the deadline but prior to the start of the allocation cycle i.e. in Dec and June every year. 

Renewal: Storage allocations can be renewed annually. Renewals require a progress report of prior award and other information described in item 6 of requirements above.

Apply here