Knowledgebase: Data Recipes > URS
How To Access URS-gated Data With cURL And Wget
Posted by Michael McEniry, Last modified by Michael McEniry on 15 December 2014 04:09 PM

When using cURL or Wget to access data over HTTP from a web server configured for URS authentication.

Step-by-step guide

  1. Make sure you have granted user profile access permissions to the website your are downloading data from. The website should have instructions on how to do this.

  2. Configure your username and password for authentication using a .netrc file

      > cd ~
      > touch .netrc
      > echo "machine login <uid> password <password>" > .netrc
      > chmod 0600 .netrc

    where <uid> is your URS username and <password> is your URS password.

  3. Create a cookie file. This will be used to persist sessions across individual cURL/Wget calls, making it more efficient.

      > cd ~
      > touch .urs_cookies

  4. Download your data.

    Using cURL:

      > curl -b ~/.urs_cookies -c ~/.urs_cookies -L -n http://server/path

    or Wget:

      > wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies http://server/path

    Note that you may supply additional options to control the output location.


(originally from

(2 vote(s))
Not helpful

Comments (0)