Useful Trick with Linux Command

Notes of some useful Linux command usage I encountered.

1. Download Google Drive files with wget

Example Google Drive shared file:

https://drive.google.com/open?id=[ThisIsFileID]

---

For general usage (not a big file)

Example command:

wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=FILEID' -O "**FILENAME**"

Where, the FILEID is the [ThisISFiledID] shwon above.

---

Download big files

Command for download any big file from google drive (for big file we need confirm download)

wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget \
    --quiet \
    --save-cookies /tmp/cookies.txt \ 
    --keep-session-cookies \
    --no-check-certificate 'https://docs.google.com/uc?export=download&id=FILEID' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=FILEID" -O "FILENAME" && rm -rf /tmp/cookies.txt

Also, subsititute FILEID and FILENAME. Note that tere are 2 FILEID.

Reference: https://gist.github.com/iamtekeste/3cdfd0366ebfd2c0d805

---

2. Download files from the requested login page with wget

Example:

Download HumanEvaI data on its official website, http://humaneva.is.tue.mpg.de/datasets_human_1.

---

Two steps:

1. Log in to the server. This only needs to be done once.

   wget --save-cookies cookies.txt \
       --keep-session-cookies \
       --post-data 'user=foo&password=bar' \
       --delete-after \
       "http://humaneva.is.tue.mpg.de/sessions"

   Make sure the --post-data parameter is properly percent-encoded (especially ampersands!) or the request will probably fail. Also make sure that user and password are the correct keys; you can find out the correct keys by sleuthing the HTML of the login page (look into your browser’s “inspect element” feature and find the name attribute on the username and password fields).

   Thus, the --post-data parameter for the HumanEvaI website should be: --post-data 'login=foo&password=bar' \

2. Now grab the page or pages we care about.

   wget --load-cookies cookies.txt \
       http://server.com/interesting/article.php

Reference: https://stackoverflow.com/questions/1324421/how-to-get-past-the-login-page-with-wget

---