This is what I’m mostly using :

I’m creating a PHP function to connect using CURL:

function get_file_curl($url,$proxy_ip,$proxy_port,$loginpassw)
{

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy_port);
curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP');
curl_setopt($ch, CURLOPT_PROXY, $proxy_ip);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $loginpassw);
$data = curl_exec($ch);
curl_close($ch);

return $data;
}

PHP cURL functions used

  • curl_init – initializes a cURL session.
  • curl_setopt – sets and option for a cURL transfer.
  • curl_exec – performs a cURL session.
  • curl_getinfo – gets information about the last transfer.
  • curl_error – returns a string containing the last error for the current session.
  • curl_close – close a cURL session.

curl_setopt options used

  • CURLOPT_URL – the URL to scrap.
  • CURLOPT_HEADER – inlude/exclude the header?
  • CURLOPT_RETURNTRANSFER – return the transfer as a string or output it out directly? Use 1, i.e. return.
  • CURLOPT_PROXY – the HTTP proxy to tunnel request through.
  • CURLOPT_HTTPPROXYTUNNEL – tunnel through a given HTTP proxy? Use 1, i.e. tunnel.
  • CURLOPT_CONNECTTIMEOUT – it’s obvious.
  • CURLOPT_REFERER – header to be used in a HTTP request.
  • CURLOPT_USERAGENT – “User Agent:” to be used in a HTTP request.

Have fun!

0 Shares:
Leave a Reply

Your email address will not be published. Required fields are marked *

18 + four =

This site uses Akismet to reduce spam. Learn how your comment data is processed.

You May Also Like

Genghis – A Single File MongoDB Admin

In order to manage MongoDB databases easily, Genghis, a single-file, self-hosted and web-based solution is pretty handy. It can be installed either as a Ruby gem or as a standalone PHP script.