Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 5 of 5
  1. #1
    New Coder
    Join Date
    Jul 2011
    Location
    USA
    Posts
    39
    Thanks
    0
    Thanked 1 Time in 1 Post

    Stop Multiple Entries on a Flat File

    Hey Guys, I subscribe to a free cron service that hits my server load page once an hour, 24 hours a day, but sometimes the bot hits it twice with just a few seconds in between. Every time it hits my page I grab the server load, date and time and write it to a flat file. For the most part it only pings it once an hour but sometimes I'll get two or three entries right on top of each other. At the end of the last month it was really bad:

    This is normal:
    Code:
    Sat 30th Jun, 12:29:26 am | Server Load:*5.76
    Sat 30th Jun, 01:29:25 am | Server Load:*4.64
    Sat 30th Jun, 02:29:25 am | Server Load:*3.51
    Sat 30th Jun, 03:29:25 am | Server Load:*3.91
    Sat 30th Jun, 04:29:24 am | Server Load:*4.31
    Sat 30th Jun, 05:29:25 am | Server Load:*5.24
    Sat 30th Jun, 06:29:25 am | Server Load:*4.50
    Sat 30th Jun, 07:29:24 am | Server Load:*4.85
    Sat 30th Jun, 08:29:29 am | Server Load:*5.59
    Sat 30th Jun, 09:29:26 am | Server Load:*4.80
    Sat 30th Jun, 10:29:24 am | Server Load:*6.23
    Sat 30th Jun, 11:29:26 am | Server Load:*6.06
    Sat 30th Jun, 12:29:26 pm | Server Load:*4.60
    Sat 30th Jun, 01:29:25 pm | Server Load:*4.45
    Sat 30th Jun, 02:29:25 pm | Server Load:*3.83
    Sat 30th Jun, 03:29:25 pm | Server Load:*4.18
    Sat 30th Jun, 04:29:24 pm | Server Load:*3.36
    Sat 30th Jun, 05:29:58 pm | Server Load:*4.11
    Then it went crazy:
    Code:
    Sat 30th Jun, 05:30:55 pm | Server Load:*3.98
    Sat 30th Jun, 05:31:52 pm | Server Load:*3.94
    Sat 30th Jun, 05:32:49 pm | Server Load:*3.85
    Sat 30th Jun, 05:34:52 pm | Server Load:*3.74
    Sat 30th Jun, 05:37:11 pm | Server Load:*3.97
    Sat 30th Jun, 05:38:07 pm | Server Load:*3.96
    Sat 30th Jun, 05:39:08 pm | Server Load:*3.83
    Sat 30th Jun, 05:39:51 pm | Server Load:*3.76
    Sat 30th Jun, 05:40:56 pm | Server Load:*3.67
    Sat 30th Jun, 05:41:54 pm | Server Load:*3.90
    Sat 30th Jun, 05:43:06 pm | Server Load:*3.76
    Sat 30th Jun, 05:45:09 pm | Server Load:*3.54
    Sat 30th Jun, 05:47:13 pm | Server Load:*3.65
    Sat 30th Jun, 05:48:05 pm | Server Load:*3.60
    Sat 30th Jun, 05:49:18 pm | Server Load:*3.49
    Sat 30th Jun, 05:50:12 pm | Server Load:*3.43
    Sat 30th Jun, 05:51:09 pm | Server Load:*3.33
    Sat 30th Jun, 05:51:58 pm | Server Load:*3.52
    Sat 30th Jun, 05:53:02 pm | Server Load:*3.49
    Sat 30th Jun, 05:53:55 pm | Server Load:*3.53
    Sat 30th Jun, 05:54:52 pm | Server Load:*3.47
    Sat 30th Jun, 05:56:10 pm | Server Load:*3.32
    Sat 30th Jun, 05:57:00 pm | Server Load:*3.38
    Sat 30th Jun, 05:58:07 pm | Server Load:*3.34
    Sat 30th Jun, 05:59:09 pm | Server Load:*3.44
    Sat 30th Jun, 05:59:33 pm | Server Load:*3.38
    Sat 30th Jun, 06:00:53 pm | Server Load:*3.27
    Sat 30th Jun, 06:01:43 pm | Server Load:*3.42
    Sat 30th Jun, 06:02:40 pm | Server Load:*3.33
    Sat 30th Jun, 06:03:37 pm | Server Load:*3.28
    Sat 30th Jun, 06:04:39 pm | Server Load:*3.20
    Sat 30th Jun, 06:05:38 pm | Server Load:*3.20
    Sat 30th Jun, 06:06:34 pm | Server Load:*3.18
    Sat 30th Jun, 06:07:37 pm | Server Load:*3.14
    Sat 30th Jun, 06:08:44 pm | Server Load:*3.06
    Sat 30th Jun, 06:29:44 pm | Server Load:*3.30
    Sat 30th Jun, 07:29:29 pm | Server Load:*2.66
    Sat 30th Jun, 08:29:31 pm | Server Load:*2.58
    Sat 30th Jun, 09:29:34 pm | Server Load:*3.32
    Sat 30th Jun, 10:29:33 pm | Server Load:*6.43
    Sat 30th Jun, 11:29:31 pm | Server Load:*3.59
    What can I do once the cron service hits my page to keep it from making a double entry (or multi entry) until the next hour?

  • #2
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,994
    Thanks
    4
    Thanked 2,662 Times in 2,631 Posts
    PHP wise whatever script you are using to log these entries can open the file first, read the last entry and if its been within the past 50 minutes or so then skip the log. That of course is only useful if its PHP doing the work. Make sure you record elsewhere that its been skipped though, otherwise you won't know there is a problem.

    Why not just use your own cron service?

  • #3
    New Coder
    Join Date
    Jul 2011
    Location
    USA
    Posts
    39
    Thanks
    0
    Thanked 1 Time in 1 Post
    Here's what I'm using to detect the cron bot.

    PHP Code:
    ######### If SetCronJob add stats #########
    $ip $_SERVER['REMOTE_ADDR'];
    $agent $_SERVER['HTTP_USER_AGENT'];
    $host gethostbyaddr($ip);
    if (
    $agent == "SetCronJob/1.0 (+http://www.setcronjob.com/)")
    {

    ######### write stats to data file #########
    $month date("F_Y");
    $data_file "server_data/server_data_$month.dat";

    $time date("D dS M, h:i:s a");
    $today date("D");
    $stats "<li class=\"$today\">$time | Server Load: $loadavg</li>\n";
    file_put_contents($data_file$statsFILE_APPEND);
    ######### end stats #########


    Quote Originally Posted by Fou-Lu View Post
    read the last entry and if its been within the past 50 minutes or so then skip the log.
    What would be the best way to do this?

    Why not just use your own cron service?
    I'm hosting with a free web host and they only allow two cron jobs per day.

  • #4
    New Coder
    Join Date
    Jul 2011
    Location
    USA
    Posts
    39
    Thanks
    0
    Thanked 1 Time in 1 Post
    Thanks for the time idea. This seems to work perfectly:

    PHP Code:
    if ( is_file($data_file) )
    {
    $age time()-filemtime($data_file);

    $min 5;
    $expire 60*$min;

    if ( 
    $age $expire )
    {
    file_put_contents($data_file$statsFILE_APPEND);
    }


  • #5
    God Emperor Fou-Lu's Avatar
    Join Date
    Sep 2002
    Location
    Saskatoon, Saskatchewan
    Posts
    16,994
    Thanks
    4
    Thanked 2,662 Times in 2,631 Posts
    Ah yeah I didn't even think about that.
    Pulling the modified time is much easier and faster than reading the file for the last entry.


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •