If you want to scrap this board, there are a lot of resources on the internet, not least on YouTube that explains how to do it. (Php) cURL is one technology that you can use. So go to YouTube and start your query: php curl and you and find videos like this
http://www.youtube.com/watch?v=lJqnGnnnwUQ
You need may need the grab_page() function.
postspam.php
Code: Select all
function login($url,$data){
$fp = fopen("cookie.txt", "w");
fclose($fp);
$login = curl_init();
curl_setopt($login, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($login, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($login, CURLOPT_TIMEOUT, 40000);
curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($login, CURLOPT_URL, $url);
curl_setopt($login, CURLOPT_HEADER, TRUE);
curl_setopt($login, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($login, CURLOPT_POST, TRUE);
curl_setopt($login, CURLOPT_POSTFIELDS, $data);
ob_start();
$data = curl_exec($login);
ob_end_clean();
curl_close($login);
unset($login);
return $data;
}
function grab_page($site){
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_TIMEOUT, 40);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_URL, $site);
ob_start();
$data = curl_exec($ch);
ob_end_clean();
curl_close($ch);
unset($ch);
return $data;
}
function post_data($site,$data){
$datapost = curl_init();
$headers = array("Expect:");
curl_setopt($datapost, CURLOPT_URL, $site);
curl_setopt($datapost, CURLOPT_TIMEOUT, 40000);
curl_setopt($datapost, CURLOPT_HEADER, TRUE);
curl_setopt($datapost, CURLOPT_HTTPHEADER, $headers);
curl_setopt($datapost, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($datapost, CURLOPT_POST, TRUE);
curl_setopt($datapost, CURLOPT_POSTFIELDS, $data);
curl_setopt($datapost, CURLOPT_COOKIEFILE, "cookie.txt");
ob_start();
$data = curl_exec($datapost);
ob_end_clean();
curl_close($datapost);
unset($datapost);
return $data;
}
You find the explanation of how to use the functions in this http://www.hackforums.net/showthread.php?tid=2977255 forum post. If you get this
error and don't want to register there, hereYou are either not logged in or do not have permission to view this page.
http://dinitside.no/temp/ReadMe.html
is the post. That files is not scrapped by a bot. I only used past special in DreamWeaver CS6 and kept all formatting. If you need to scrape the whole board, you should definitely by the book: http://webbotsspidersscreenscrapers.com/
2. Posting spam.
This board is closed so you can not post spam. That is the main reason the board was closed down. If you need my personal advice, you have to pay me USD 500/hour. Wages are high in Norway and I have a solid background and theoretical education, at least on the master level i mathematics, statistics, economics and information technology. I have education on the Dr Scient level in numerical analysis and mathematical finance. And I don't live far from Sweden where the excellent cURL library was made. Like me they have a background in assembly so don't be surprised if I end up spamming and hacking your company or computer:-)
3. Related links
fetch all youtube videos using curl
A good web data extraction/screen scraper program?
How do screen scrapers work?