This is what I’m mostly using :
I’m creating a PHP function to connect using CURL:
function get_file_curl($url,$proxy_ip,$proxy_port,$loginpassw) { $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_PROXYPORT, $proxy_port); curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP'); curl_setopt($ch, CURLOPT_PROXY, $proxy_ip); curl_setopt($ch, CURLOPT_PROXYUSERPWD, $loginpassw); $data = curl_exec($ch); curl_close($ch); return $data; }
PHP cURL functions used
- curl_init – initializes a cURL session.
- curl_setopt – sets and option for a cURL transfer.
- curl_exec – performs a cURL session.
- curl_getinfo – gets information about the last transfer.
- curl_error – returns a string containing the last error for the current session.
- curl_close – close a cURL session.
curl_setopt options used
- CURLOPT_URL – the URL to scrap.
- CURLOPT_HEADER – inlude/exclude the header?
- CURLOPT_RETURNTRANSFER – return the transfer as a string or output it out directly? Use 1, i.e. return.
- CURLOPT_PROXY – the HTTP proxy to tunnel request through.
- CURLOPT_HTTPPROXYTUNNEL – tunnel through a given HTTP proxy? Use 1, i.e. tunnel.
- CURLOPT_CONNECTTIMEOUT – it’s obvious.
- CURLOPT_REFERER – header to be used in a HTTP request.
- CURLOPT_USERAGENT – “User Agent:” to be used in a HTTP request.
Have fun!