Data File Transfer Methods

To submit datasets to the GRIIDC data management system, a dataset must first be identified through an approved Dataset Information Form (DIF). The dataset can then be submitted to GRIIDC through Dataset Submission. Dataset Submission allows you to submit a dataset file to the GRIIDC system using different methods. The method you will use depends on the size of the dataset file and your technical abilities and experience. For a dataset file that is less than 2 GB in size, GRIIDC recommends using the Direct Upload method to submit your file; this method does not require any specialized software or technical expertise. Other methods to submit a dataset file to GRIIDC are described below. Please contact GRIIDC if you have questions or concerns about what method you should use to submit your dataset.

HTTP (website)
HTTP can be used to pull a dataset file that has been submitted to a National Data Archive Center or point to a dataset that is available on a public website. Alternatively, your institution may have a public HTTP site where you can place your dataset file. Contact your institutional IT department to determine if a public HTTP server is available for this purpose. Due to differences in institutional policies and procedures, GRIIDC is unable to assist users in establishing websites and HTTP servers or provide support for these servers once established.

Once your dataset file is on a website, visit Dataset Submission. Select the “Request Pull from HTTP/FTP Server” tab under the Dataset File Transfer Details section. Please provide the web address (URL) in the “Dataset File URL” field. Select “Submit” from the bottom of the screen and your dataset will be submitted to the GRIIDC data management system.

FTP
FTP can be used to pull a dataset file through a FTP server. This will usually be a FTP server hosted by the submitter's institution. Due to differences in institutional policies and procedures, GRIIDC is unable to set up FTP servers for data providers or provide support for these servers once they are established. Please contact your institution’s IT department to determine your institutional policies regarding FTP and to request assistance in setting up a FTP server.

Once your dataset file is on the FTP server, visit Dataset Submission. Under the Submit section, select the “Request Pull from HTTP/FTP Server” tab under the Dataset File Transfer Details section. Please provide the web address (URL) in the “Dataset File URL” field. Select “Submit” from the bottom of the screen and your dataset will be submitted to the GRIIDC data management system.

SFTP
SFTP is a method to submit a dataset file to GRIIDC when it is not possible or practical for you to provide your dataset file by direct transfer, a website, or an FTP server, if the dataset file is more than 2GB and you are able to establish a SFTP client. Due to differences in institutional policies and procedures, GRIIDC is unable to set up SFTP clients for data providers or provide support for these clients once they are established. Please contact your institution’s IT department to determine your institutional policies regarding SFTP and to request assistance in setting up a SFTP client.

Some examples of popular free to use SFTP clients are listed here. Please note this is list is not exhaustive and should not be considered an endorsement by GRIIDC:

In order to transfer a dataset file via SFTP you must configure your account. To do so visit Dataset Submission and after completing all required fields, select the Submit tab. Select the "Upload via SFTP/GridFTP" tab and select the button REQUEST SFTP/GridFTP. Your account will automatically be configured.

To configure your SFTP client to the GRIIDC SFTP client, please use the following parameters:

  • Host Name: data.gomri.org
  • Port: 22

To log in, please use your GRIIDC account credentials (username and password).

Upon connecting to GRIIDC SFTP you will be in your home directory (/). This directory is read only.

In order to upload a dataset file, please change directory (cd) to the “incoming” directory. This directory has full write access and is where you can deposit your file.

Once you have transferred your file to the GRIIDC SFTP client, you must submit the data file using Dataset Submission. Under the Submit section, select the "Upload via SFTP/GridFTP" tab under Dataset File Transfer Details. Select “Browse” under “Dataset File Path” and choose your dataset file from the incoming directory. Once your file has been selected the window will automatically close. Then select “Submit” from the bottom of the screen and your dataset will be submitted to the GRIIDC data management system.

GridFTP
GridFTP is a method to submit data to GRIIDC when it is not possible or practical for you to provide data by another method, if the file size is more than 2 GB and you are able to establish a GridFTP endpoint.

You will require a Globus account (www.globus.org) to use GridFTP. If you are setting up GridFTP on a personal computer, Globus Personal Connect software can be used to establish a GridFTP endpoint on your personal computer. GridFTP can also be used to conduct server-to-server data transfers, requiring Globus Connect Server Software. Due to differences in institutional policies and procedures, GRIIDC is unable to set up GridFTP endpoints for data providers or provide technical support once an endpoint is established. Please contact your institution’s IT department to determine your institutional policies regarding GridFTP and to request assistance in setting up a GridFTP endpoint.

In order to transfer a dataset file via GridFTP you must configure your account. To do so visit Dataset Submission and once you have completed all required fields select the Submit tab. In the Submit section select the “Upload via SFTP/GridFTP” tab and select the button “REQUEST SFTP/GridFTP”.Your account will automatically be configured.

To transfer your dataset file, log in to the globus.org interface and activate your personal endpoint and the gomri#griidc endpoint. You will be prompted to authenticate on the endpoint server. Please use your GRIIDC account credentials (username and password) to authenticate. On the gomri#griidc endpoint select the “incoming” folder and queue the transfer of the appropriate file. Once your file is transferred, you must visit the Dataset Submission page. Under the Submit section, select the “Upload via SFTP/GridFTP” tab under Dataset File Transfer Details. Select “Browse” under “Dataset File Path”. You can then select your dataset file from the appropriate directory. Once you select your dataset file the window will close. Select “Submit” from the bottom of the screen and the dataset file will be submitted to the GRIIDC system.