Knowledgebase: Data Recipes > URS
How to use cURL and Wget with Earthdata Login (URS)
Posted by Michael McEniry, Last modified by Michael McEniry on 10 July 2017 02:25 PM

When using cURL or Wget to access data over HTTP from a web server configured for URS authentication.

Note: Starting with version 1.11.1, Wget requires the "--auth-no-challenge" option to authenticate properly to URS. Versions prior to 1.11 should omit the "--auth-no-challenge" option. Version 1.11 of Wget will not work at all.

Step-by-step guide

  1. Make sure you have granted user profile access permissions to the website your are downloading data from. The website should have instructions on how to do this.

  2. Configure your username and password for authentication using a .netrc file

    > cd ~
    > touch .netrc
    > echo "machine urs.earthdata.nasa.gov login <uid> password <password>" > .netrc
    > chmod 0600 .netrc

    where <uid> is your URS username and <password> is your URS password.

  3. Create a cookie file. This will be used to persist sessions across individual cURL/Wget calls, making it more efficient.

    > cd ~
    > touch .urs_cookies

  4. Download your data.

    Using cURL:

    > curl -b ~/.urs_cookies -c ~/.urs_cookies -L -n http://server/path

    or Wget:

    > wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge http://server/path

    Note that you may supply additional options to control the output location.

 

(originally from https://wiki.earthdata.nasa.gov/display/URSFOUR/How+To+Access+Data+With+cURL+And+Wget)

(4 vote(s))
Helpful
Not helpful

Comments (0)

*******************************************************************************
By accessing and using this information system, you acknowledge consent to the following:
You are accessing a U.S. Government information system, which includes:
(1) this computer; (2) this computer network; (3) all computers connected to this network including end user systems; (4) all devices and storage media attached to this network or to any computer on this network; and (5) cloud and remote information services.
This information system is provided for U.S. Government-authorized use only. You have no reasonable expectation of privacy regarding any communication transmitted through or data stored on this information system. At any time, and for any lawful purpose, the U.S. Government may monitor, intercept, search, and seize any communication or data transiting, stored on, or traveling to or from this information system. You are NOT authorized to process classified information on this information system. Unauthorized or improper use of this system may result in suspension or loss of access privileges, disciplinary action, and civil and/or criminal penalties.

*******************************************************************************