How to hack this board? Please crawl me.

With a focus on cURL
Site Admin
Posts: 178
Joined: Tue Sep 29, 2009 6:25 pm
Location: Moss Norway

How to hack this board? Please crawl me.

Postby KBleivik » Sat Jun 22, 2013 9:29 am

1. Crawl the board, but don't eat up all bandwidth.

If you want to scrap this board, there are a lot of resources on the internet, not least on YouTube that explains how to do it. (Php) cURL is one technology that you can use. So go to YouTube and start your query: php curl and you and find videos like this

You need may need the grab_page() function.


Code: Select all

function login($url,$data){
    $fp = fopen("cookie.txt", "w");
    $login = curl_init();
    curl_setopt($login, CURLOPT_COOKIEJAR, "cookie.txt");
    curl_setopt($login, CURLOPT_COOKIEFILE, "cookie.txt");
    curl_setopt($login, CURLOPT_TIMEOUT, 40000);
    curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
    curl_setopt($login, CURLOPT_URL, $url);
    curl_setopt($login, CURLOPT_HEADER, TRUE);
    curl_setopt($login, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
    curl_setopt($login, CURLOPT_POST, TRUE);
    curl_setopt($login, CURLOPT_POSTFIELDS, $data);
    $data = curl_exec($login);
    return $data; 

function grab_page($site){
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
    curl_setopt($ch, CURLOPT_TIMEOUT, 40);
    curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
    curl_setopt($ch, CURLOPT_URL, $site);
    $data = curl_exec($ch);
    return $data; 

function post_data($site,$data){
    $datapost = curl_init();
    $headers = array("Expect:");
    curl_setopt($datapost, CURLOPT_URL, $site);
    curl_setopt($datapost, CURLOPT_TIMEOUT, 40000);
    curl_setopt($datapost, CURLOPT_HEADER, TRUE);
    curl_setopt($datapost, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($datapost, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    curl_setopt($datapost, CURLOPT_POST, TRUE);
    curl_setopt($datapost, CURLOPT_POSTFIELDS, $data);
        curl_setopt($datapost, CURLOPT_COOKIEFILE, "cookie.txt");
    $data = curl_exec($datapost);
    return $data; 

You find the explanation of how to use the functions in this forum post. If you get this

You are either not logged in or do not have permission to view this page.

error and don't want to register there, here

is the post. That files is not scrapped by a bot. I only used past special in DreamWeaver CS6 and kept all formatting. If you need to scrape the whole board, you should definitely by the book:

2. Posting spam.

This board is closed so you can not post spam. That is the main reason the board was closed down. If you need my personal advice, you have to pay me USD 500/hour. Wages are high in Norway and I have a solid background and theoretical education, at least on the master level i mathematics, statistics, economics and information technology. I have education on the Dr Scient level in numerical analysis and mathematical finance. And I don't live far from Sweden where the excellent cURL library was made. Like me they have a background in assembly so don't be surprised if I end up spamming and hacking your company or computer:-)

3. Related links

fetch all youtube videos using curl

A good web data extraction/screen scraper program?

How do screen scrapers work?

Return to “URL solutions”

Who is online

Users browsing this forum: No registered users and 1 guest