Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Page 1 of 2 12 LastLast
Results 1 to 15 of 20
  1. #1
    Regular Coder mOrloff's Avatar
    Join Date
    Nov 2008
    Location
    The Great Pacific NW, USA
    Posts
    424
    Thanks
    8
    Thanked 6 Times in 6 Posts

    Script is maxing out memory. WHY ???? (painfully urgent)

    I have a script which was working wonderfully for a few weeks, then suddenly it started maxing out the memory.
    We bumped up the memory_limit in the php.ini, it worked for a wink, then re-exibited the same symptoms. Just to crutch it along, I re-upped the mem limit to 128m, and it didn't take long to hit that.

    I don't think anyone changed anything in the script, and the only change I can come up with (and it doesn't even seem all that related to me) is that I added another Table to one of the DB's which this script accesses (prior to this, it was a single-table db).

    I don't make any SELECT * queries.
    I only select the columns which I need.

    What are some likely culprits ??
    Where should I look ??

    HELP !!

    FYI, here's the error from the log:
    [04-Mar-2010 22:41:31] PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 74 bytes) in blah/blah/script.php on line 147
    The line referenced there is simply the opening of a while loop
    PHP Code:
    while($row mysql_fetch_assoc($result)) { 
    If you would like additional info, please just let me know.

    Edit:
    The script is over 300 lines of code so I'm hoping to see if there are any "usual suspects" to eliminate prior to asking anyone to go through all that.


    ~ Mo

  • #2
    Regular Coder
    Join Date
    Jan 2009
    Posts
    193
    Thanks
    0
    Thanked 20 Times in 20 Posts
    Try checking the query that you are trying to run on that line. If you have phpmyadmin or any other query browser software then try adding EXPLAIN to the front of the query to figure out what the query is doing.

    Example
    Code:
    EXPLAIN SELECT id, name, other_field FROM mytable WHERE ....
    If you don't understand what the explain is returning when you run it then take a screenshot or if you can copy and paste the info. It will look something like the image below.
    Last edited by skywalker2208; 03-05-2010 at 12:38 AM. Reason: Added image

  • #3
    Regular Coder
    Join Date
    Dec 2009
    Location
    UK
    Posts
    495
    Thanks
    0
    Thanked 58 Times in 58 Posts
    The number of lines of code are not important really. The real question is why do you need to select out all this data at once, is there a more streamlined query you can use instead. Also, have you considered paginating your output (if outputting large amounts of results). The full script would be beneficial (up to that point at least)
    My site: JayGilford.com
    Resources:
    PHP Pagination Class | Getting all page links | Handling PHP Errors properly
    If you like a users help, show your appreciation with the rep and thanks buttons :)

  • #4
    Regular Coder mOrloff's Avatar
    Join Date
    Nov 2008
    Location
    The Great Pacific NW, USA
    Posts
    424
    Thanks
    8
    Thanked 6 Times in 6 Posts
    Thanks-a-BUNCH for your responses.
    I didn't realize how much time I had burned trying to get it going, and I had to leave for an appointment.
    It looks like the sales manager just has to wait until the morning now (My development machine is at the office)

    I'll run your suggestion first thing, and post my position.

    ~ Mo
    Last edited by mOrloff; 03-05-2010 at 01:55 AM.

  • #5
    Regular Coder mOrloff's Avatar
    Join Date
    Nov 2008
    Location
    The Great Pacific NW, USA
    Posts
    424
    Thanks
    8
    Thanked 6 Times in 6 Posts
    Quote Originally Posted by JAY6390 View Post
    ... why do you need to select out all this data at once, is there a more streamlined query you can use instead? ...
    Only pertinent records are being grabbed (not all the data at once), and this is how:
    PHP Code:
    $fields='pn, AVG(pricing) AS pricing'// query component: all the needed columns
    $table='vend1'// query component: look in this table
    $condition='pn LIKE "'.$word.'%"'// query component: get anything that starts with the word
    $orderBy=NULL// query component
    $groupBy='pn'// query component
    $result=qryFunc($fields,$table,$condition,$orderBy,$groupBy); // run it again to include any partial matches
    ...
    while(
    $row mysql_fetch_assoc($result)) ... 
    Last edited by mOrloff; 03-05-2010 at 03:48 PM.

  • #6
    Regular Coder mOrloff's Avatar
    Join Date
    Nov 2008
    Location
    The Great Pacific NW, USA
    Posts
    424
    Thanks
    8
    Thanked 6 Times in 6 Posts
    Alright.
    I ran this query:
    PHP Code:
    SELECT pnAVG(pricing) AS pricing
    FROM vend1
    WHERE pn LIKE 
    "1n4148%"
    GROUP BY pn 
    And it went smooth as silk. 28 records were returned.

    Here's a screencap from the EXPLAIN:


    Any ideas on what could be chewing through sooo much ram??

    ~ Mo

  • #7
    Regular Coder
    Join Date
    Dec 2009
    Location
    UK
    Posts
    495
    Thanks
    0
    Thanked 58 Times in 58 Posts
    To use that amount of memory, you're either loading in a lot of data, or looping excessively. As I originally said, it would be easier if you posted the code to diagnose this. The code you have provided so far is highly vague and not of any great use on its own
    My site: JayGilford.com
    Resources:
    PHP Pagination Class | Getting all page links | Handling PHP Errors properly
    If you like a users help, show your appreciation with the rep and thanks buttons :)

  • #8
    Supreme Master coder! abduraooft's Avatar
    Join Date
    Mar 2007
    Location
    N/A
    Posts
    14,864
    Thanks
    160
    Thanked 2,224 Times in 2,211 Posts
    SELECT pn, AVG(pricing) AS pricing
    FROM vend1
    WHERE pn LIKE "1n4148%"
    GROUP BY pn
    Since you are using LIKE to match the records, mysql can't use any indexes and thus it loops though all the 716166 records in your table. This might be the cause. A good DB design and proper indexing would be the only solution.
    The Dream is not what you see in sleep; Dream is the thing which doesn't let you sleep. --(Dr. APJ. Abdul Kalam)

  • #9
    Regular Coder mOrloff's Avatar
    Join Date
    Nov 2008
    Location
    The Great Pacific NW, USA
    Posts
    424
    Thanks
    8
    Thanked 6 Times in 6 Posts
    Quote Originally Posted by JAY6390 View Post
    ... it would be easier if you posted the code to diagnose this...
    Alright, you asked for it
    If there aren't any usual suspects for me to knock off first, then I guess the next step is to lay it ALL out for you guys.

    This is a script to which I have emails piped.
    It chew through the subject and msg grabbing anything that has potential to be a part number, then compares those strings to a lexicon of part numbers.
    For those which come back with a match prices are then gathered, and a quote email is returned to the sender.

    I tried to attach the file, but the "attachments" feature isn't working for some reason, so I zipped it and posted it for download here:
    http://www.voyagercomponents.com/download/quoter.zip

    Thanks-a-bunch.

  • #10
    Regular Coder mOrloff's Avatar
    Join Date
    Nov 2008
    Location
    The Great Pacific NW, USA
    Posts
    424
    Thanks
    8
    Thanked 6 Times in 6 Posts
    Quote Originally Posted by abduraooft View Post
    ... using LIKE ... might be the cause. A good DB design and proper indexing would be the only solution.
    Thanks, I'll see what I can figure out.

    Would that problem match the symptoms where it evidently just keeps adding to the memory storage ??
    It runs successfully several times until it finally hits the limit, and it was running on 32 MB for weeks, then suddenly maxed it out.
    Then, maxed out 64MB after just being run a few times, then 128MB after just another handful of run cycles.

    ???

    Thanks-a-bunch,
    ~ Mo
    Last edited by mOrloff; 03-05-2010 at 06:08 PM.

  • #11
    UE Antagonizer Fumigator's Avatar
    Join Date
    Dec 2005
    Location
    Utah, USA, Northwestern hemisphere, Earth, Solar System, Milky Way Galaxy, Alpha Quadrant
    Posts
    7,691
    Thanks
    42
    Thanked 637 Times in 625 Posts
    Are you using persistent connections to MySQL?

  • #12
    Regular Coder mOrloff's Avatar
    Join Date
    Nov 2008
    Location
    The Great Pacific NW, USA
    Posts
    424
    Thanks
    8
    Thanked 6 Times in 6 Posts
    You know, I'm not sure.
    How do I find out?

    ~ Mo

  • #13
    UE Antagonizer Fumigator's Avatar
    Join Date
    Dec 2005
    Location
    Utah, USA, Northwestern hemisphere, Earth, Solar System, Milky Way Galaxy, Alpha Quadrant
    Posts
    7,691
    Thanks
    42
    Thanked 637 Times in 625 Posts
    You would be calling mysql_pconnect() rather than mysql_connect().

  • #14
    Regular Coder mOrloff's Avatar
    Join Date
    Nov 2008
    Location
    The Great Pacific NW, USA
    Posts
    424
    Thanks
    8
    Thanked 6 Times in 6 Posts
    In that case, no I am not.

  • #15
    UE Antagonizer Fumigator's Avatar
    Join Date
    Dec 2005
    Location
    Utah, USA, Northwestern hemisphere, Earth, Solar System, Milky Way Galaxy, Alpha Quadrant
    Posts
    7,691
    Thanks
    42
    Thanked 637 Times in 625 Posts
    That's a puzzler... PHP does its own garbage collection and each request is independant of the one before so why wouldn't the memory be freed?

    If you put a counter in your while() loop and echo it, how many times does it loop through and fetch data before it errors out?

    (Edit) One other question, is this quoter.php called inside a loop from another script? It just sounds like a runaway script to me but I can't really see why it would be running away based on the code.
    Last edited by Fumigator; 03-05-2010 at 06:58 PM.


  •  
    Page 1 of 2 12 LastLast

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •