Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Page 1 of 2 12 LastLast
Results 1 to 15 of 16
  1. #1
    New Coder
    Join Date
    Nov 2011
    Posts
    54
    Thanks
    4
    Thanked 0 Times in 0 Posts

    Most reliable way of retrieving file from remote server?

    Hi all - I am writing a PHP script that will be installed on a users server. I want to pull a file from MY remote server (a zip file) and write it to the local server that the script is running on.

    I can do it just fine use file_get_contents and also using Snoopy, fsockopen or CURL.

    However, the issue is that my script will be distributed to lots of people who will have a variety of servers.
    They will range from freebie webspace to their own dedicated rack with everything in between
    I can't be certain that the correct options will be enabled for file_get_contents/Snoopy/fsock/CURL to work and I know that, if certain options are disabled, it will be unlikely that the user will be able to get them enabled.

    So, I am thinking that, pretty much, the only thing I can guarantee is the they will have PHP 4 + what is the best way to pull the remote file - IE: Which way has the best chance of working on such a large range of webservers :

    file_get_contents
    Snoopy (using fetch)
    fsockopen
    CURL

    Any ideas or comments would be MUCH appreciated

  • #2
    Regular Coder
    Join Date
    Sep 2007
    Posts
    264
    Thanks
    19
    Thanked 0 Times in 0 Posts
    Are you just looking at transferring file from server to another?
    connection.php
    PHP Code:
    <?php
    $server 
    "domain.com"//target server
    $user "ftpusername@domain.com"//target user
    $pass "password"//password on target server for Ftp
    ?>
    transfer.php
    PHP Code:
    <?php
    require_once('connection.php');
    $file "file.zip";
    $local_file 'local.zip';
    $sessid ftp_connect($server);
    $login_ok ftp_login($sessid$user"$pass");
    if ((!
    $sessid) || (!$login_ok)):
     echo 
    "failed to connect: check hostname, username & password";
     exit; 
    //failed? Unable to connect!
    endif;
     
    if (
    ftp_get($sessid$local_file$fileFTP_BINARY))
    {
     echo 
    "Successfully written\n";
    } else {
     echo 
    "There was a issue\n";
    }
     
    ftp_close($sessid);
    ?>

  • #3
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,994
    Thanks
    4
    Thanked 2,662 Times in 2,631 Posts
    Of the four you've posted, sockets are the most reliable. Curl is an extension and snoopy is a package. File_get_contents has only existed as of 4.3+, and will only work remotely if allow_url_fopen is enabled. Depending on what you are doing, ftp may also be an option.
    There is no need to limit yourself to one. Try by route of ease: file_get_contents, then curl, then sockets (I don't know where snoopy fits in here). If one fails, try the next until all have been exhausted.

  • #4
    Senior Coder
    Join Date
    Feb 2011
    Location
    Your Monitor
    Posts
    4,421
    Thanks
    62
    Thanked 535 Times in 522 Posts
    Quote Originally Posted by spman View Post
    if certain options are disabled, it will be unlikely that the user will be able to get them enabled.

    So, I am thinking that, pretty much, the only thing I can guarantee is the they will have PHP 4 + what is the best way to pull the remote file
    Erm, I hate to point this out but if the host has disabled remote functions, php version 4 or 5 you're still not going to be able to download a remote file.

    I'll repeat that in slightly different wording:
    It doesn't matter what version of php the server has installed. If the server administrator has disabled remote functions then you won't be able to download a remote file at all.

    I can't get my head around why you think you'll still be able to
    See my new CodingForums Blog: http://www.codingforums.com/blogs/tangoforce/

    Many useful explanations and tips including: Cannot modify headers - already sent, The IE if (isset($_POST['submit'])) bug explained, unexpected T_CONSTANT_ENCAPSED_STRING, debugging tips and much more!

  • #5
    New Coder
    Join Date
    Nov 2011
    Posts
    54
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by tangoforce View Post
    Erm, I hate to point this out but if the host has disabled remote functions, php version 4 or 5 you're still not going to be able to download a remote file.

    I'll repeat that in slightly different wording:
    It doesn't matter what version of php the server has installed. If the server administrator has disabled remote functions then you won't be able to download a remote file at all.

    I can't get my head around why you think you'll still be able to
    Hi tangoforce - thanks for the post - I realise that and, under that circumstance, I would have to resort to asking them to DL everything and upload it via FTP - I just wanted to know what the order of precedence should be for likleyhood of a particular method working.

    I guess the answer is try 'em all and if they all fail then resort to FTP.


    EDIT: Let me rephrase the question - Let's assume that remote functions are enabled (otherwise nothing but local FTP will work anyway) - In what order are the other methods likely to work (IE: what other dependencies in the server config could affect them) ?
    I am guessing that if the server admin has disabled remote function then none of the methods will work - In your experience, how common is it to have that functionality disabled?
    Last edited by spman; 08-14-2012 at 11:01 PM.

  • #6
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,994
    Thanks
    4
    Thanked 2,662 Times in 2,631 Posts
    allow_url_fopen I find is most often disabled (although somewhat rare even, its allow_url_include that should be disabled), but sockets and curl are often available. Socket's I'd deem most reliable as curl is an extension on the PHP which may or may not be compiled in.
    For ease, I'd try file_get_contents if the size is small enough, then I'd try curl, then sockets. Sockets are more complicated since you have to send the instructions for the http get/post.
    You can always use ini_get calls, function_exists calls, and extension_loaded calls prior to doing anything. This helps to determine what you can do. So if PHP version is > 4.3 and allow_url_fopen is enabled, you can use file_get_contents. If is < 4.3 (won't go back to 3.0 documentation; sockets I recall being there but can't verify as the API lists 4.0+, but I no longer deem 3.x as a common solution, only custom code I write for such legacy machines), then sockets are a good bet. If you are on 4.0.2+, but less than 4.3, then curl can be probed for extension_loaded, and go from there.

    On the other hand, you may approach it differently. Trying sockets may be the best bet as it is likely the most available feature - its built in core, not blocked by safe mode, and not a default disabled function. But it takes the most work to do what you need to do.

    So short answer is, there really isn't the best way to do this. There is just different ways to do it. I'd probably go for sockets over anything else.

  • #7
    Senior Coder
    Join Date
    Feb 2011
    Location
    Your Monitor
    Posts
    4,421
    Thanks
    62
    Thanked 535 Times in 522 Posts
    Personally, if it were me and I wanted a reliable way of doing a file download on a server that didn't have it turned off..

    1) cURL - Not guaranteed to be available
    2) file_get_contents() - as Fou says, small files only otherwise it may be unreliable
    3) sockets - most reliable but most complex to use. You will however find samples on google.
    See my new CodingForums Blog: http://www.codingforums.com/blogs/tangoforce/

    Many useful explanations and tips including: Cannot modify headers - already sent, The IE if (isset($_POST['submit'])) bug explained, unexpected T_CONSTANT_ENCAPSED_STRING, debugging tips and much more!

  • #8
    New Coder
    Join Date
    Nov 2011
    Posts
    54
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Thank you so much guys for your help!

    Another question if I may?

    The file(s) are stored currently in an Amazon S3 Bucket - so the url to access them may be something like :

    http://example.s3.amazonaws.com/files/file1.zip

    I'm guessing that with S3 it should still be possible to retrieve the file using sockets and port 80?

  • #9
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,994
    Thanks
    4
    Thanked 2,662 Times in 2,631 Posts
    Yeah, you simply issue the port connection, and then write the HTTP GET status for it. You should then read the response back; you can use it to determine the status (200 is the only valid one in http) and process the remaining results after header.

    Here's an example on the API, the first one fetches data from port 80: http://ca3.php.net/manual/en/function.fsockopen.php

  • #10
    New Coder
    Join Date
    Nov 2011
    Posts
    54
    Thanks
    4
    Thanked 0 Times in 0 Posts
    I can't believe the help you guys have been so forthcoming with! You certainly have a lot more patience then some other forums/q&a sites! So, thank you!

    If sockets are going to be the most likely to be available I guess I should try that first.

    Do I have this right in the dependancies etc :

    1. CURL - only works if the CURL extension is enabled but it doesn't matter if allow_url_fopen or allow_url_include are disabled?

    2. file_get_contents - only works if EITHER allow_url_fopen or allow_url_include are enabled ?

    3. Sockets - I have no idea what they depend on to work LOL ????

  • #11
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,994
    Thanks
    4
    Thanked 2,662 Times in 2,631 Posts
    Curl is independent of allow_url_fopen. The two allow directives are for filesystem wrappers using fopen and include.

    File_get_contents will only work if you open them in the fashion associated with the particular ini. allow_url_fopen will allow file_get_contents to work remotely. Allow_url_include has no impact on file_get_contents as its a nonparsed read of a file and not a parsed inclusion of the file. The allow_url_include is designed to include/require remote files, but IMO carries 0 value with it since you would only get the reply after it's parsed by the webhost.

    Sockets are core. They don't depends on anything beyond the socket timeout (that I'm aware of). But it is a function family, so the functions can be added to the disabled_functions directive which would forbid their usage. If these are closed, typically all of the remote calls will be.

  • #12
    New Coder
    Join Date
    Nov 2011
    Posts
    54
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Ok - I have written some test code that pulls an 18k jpg from my Amazon S3 and writes it to my local server.

    Using file_get_contents it takes seconds.

    Using sockets it takes about a minute.

    Does anyone have any ideas please why it is so much slower with sockets?

    For reference, I have used a version of the code found in example four on this page :

    http://www.php-mysql-tutorial.com/wi...using-php.aspx

    EDIT: After some googling I found a reference to a similar problem here : http://www.nutt.net/2011/12/04/fsock...y-slow-in-php/

    However, I don't really know where I should be integrating the Connection: Close into the function referenced in the other site??

    Many thanks in advance for any help
    Last edited by spman; 08-21-2012 at 09:41 PM.

  • #13
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,994
    Thanks
    4
    Thanked 2,662 Times in 2,631 Posts
    Close would replace your current connection.

  • #14
    New Coder
    Join Date
    Nov 2011
    Posts
    54
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Fou-Lu View Post
    Close would replace your current connection.
    I'm not sure what you mean i'm afraid ?

  • #15
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,994
    Thanks
    4
    Thanked 2,662 Times in 2,631 Posts
    If you've used that link to build your fsockopen, it passes a connection of keep-alive. Replace it with a connection close.

  • Users who have thanked Fou-Lu for this post:

    spman (08-22-2012)


  •  
    Page 1 of 2 12 LastLast

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •