last day (15 days later) » 

8:49 AM
1
A: Get all the URLs using multi curl

Joseph_JHere is a function that I put together that will properly utilize the curl_multi_init() function. It is more or less the same function that you will find on PHP.net with some minor tweaks. I have had great success with this. function multi_thread_curl($urlArray, $optionArray, $nThreads) { //G...

 
thanks, Ill check it.
 
Yup no problem, let me know if you have any issues.
 
I got a problem Warning: Use of undefined constant ch - assumed 'ch' (this will throw an Error in a future version of PHP) this 3 lines ${ch . $thread} = curl_init(); curl_setopt_array(${ch . $thread}, $optionArray); //Set your main curl options. curl_setopt(${ch . $thread}, CURLOPT_URL, $value); //Set url.
 
Thank you for pointing that out. I have addressed the issue. Please check updated answer.
I made some changes to the updated code. Use the current code. Sorry for the rapid changes.
 
thanks for help, sorry too, I'm the one who's late to reply.
 
8:49 AM
Yup no problem, let me know if helps you.
 
thanks, Ill apply my current code here.
 
8:59 AM
hello
 
hi,
 
So did that work for ya
 
yea, but the code I have is not working on it.
 
Sorry, I am confused
Did my solution speed up your curl
 
sorry about that, well havent tried many urls, I mean I want to get all the urls inside every site.
-1
A: Get all the URLs using multi curl

user123This is what I achieved after working on the code, It worked but not sure if this is the best answer. Kindly check my code. <?php $array = array('https://www.google.com/','https://www.google.com/','https://www.google.com/','https://www.google.com/','https://www.google.com/','https://www.google....

you can check my code
 
9:03 AM
The code in that answer is not the code that I suggested for you. Have you tried my code?
 
yea, and im trying to add my code on yours
`$urlContent = file_get_contents($url);

$dom = new DOMDocument();
@$dom->loadHTML($urlContent);
$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate("/html/body//a");

for($i = 0; $i < $hrefs->length; $i++){
$href = $hrefs->item($i);
$url = $href->getAttribute('href');
$url = filter_var($url, FILTER_SANITIZE_URL);
// validate url
if(!filter_var($url, FILTER_VALIDATE_URL) === false){
echo '<a href="'.$url.'">'.$url.'</a><br />';
}
}`
that line
 
Don't
 
is it not possible?
 
When you run my code you will get an array that contains all of the html from your url's
so what you need to do is loop over the results array and then get your urls
 
its not displaying array, its shows the content
 
9:06 AM
that is good
you have the data
so know do something like this
 
I see
 
$results = multi_thread_curl($urlArray, $optionArray, $nThreads);

foreach($results as $key){

$dom = new DOMDocument();
@$dom->loadHTML($key);
$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate("/html/body//a");

for($i = 0; $i < $hrefs->length; $i++){

$href = $hrefs->item($i);
$url = $href->getAttribute('href');
$url = filter_var($url, FILTER_SANITIZE_URL);
// validate url
if(!filter_var($url, FILTER_VALIDATE_URL) === false){
echo '<a href="'.$url.'">'.$url.'</a><br />';
}

}

}
 
wow, can you add that to your answer. so I can accept it
 
yes
Updated
 
thanks a lot
 
9:15 AM
yeah no problem! I hope it helped... Good luck on the rest of your project! ~Cheers
 

last day (15 days later) »