Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 6 of 6
  1. #1
    Regular Coder jfreak53's Avatar
    Join Date
    May 2004
    Location
    Guatemala
    Posts
    477
    Thanks
    19
    Thanked 10 Times in 10 Posts

    Many many AJAX calls, how to make it work

    I have this system that I am making and the final part of it requires that I transfer content from one system to another once a day. We are talking about somewhere around 50K sql queries (Don't ask, I didn't make it I am just moving it ). Problem is I cannot do that many in PHP without a timeout issue, not to mention the client will get tired of just sitting there waiting and the apache timeout limit will be met also.

    So I figured I'd do it with a progress bar and ajax. I got the system setup right to accept the calls and run through the script just fine, that's not a problem. The problem is the DOM freezes because my ajax calls are not Async. Here is my script:

    Code:
    function ajax ( func, params, coms ) {
    	return j.ajax({
       type:"GET",
       url: "libraries/caller.php?function=tras_"+func+"&params="+params+coms,
       async: false,
       dataType: "json"
      }).responseText;
    }
    
    function ajaxP ( func, js, params, coms ) {
    	return j.ajax({
       type:"POST",
       url: "libraries/caller.php?function=tras_"+func+"&params="+params+coms,
       async: false,
       data: js,
       dataType: "json",
       beforeSend: function(x) {
         if (x && x.overrideMimeType) {
           x.overrideMimeType("application/j-son;charset=UTF-8");
         }
       }
      }).responseText;
    }
    
    function startTras () {
    	var startDate = new Date(j('#del').val());
    	var endDate = new Date(j('#al').val());
    	var del = getEl('del');
    	var al = getEl('al');
    
    	if (startDate <= endDate){
    		j('#main').html('<p><div id="fech"></div><div id="mes"></div><div id="error"></div></p><div id="progressbar" style="width: 200px; height: 22px;"></div>');
    		a = ajax('checkInfo', '', '&del='+del+'&al='+al);
    		s = j.parseJSON(a);
    		if(s.err > '') {
    			j('#err').html('Error, no se puede continuar...<br>'+s.err);
    			j('#progressbar').progressbar({ value: 100 });
    		} else {
    			var curdat = s.params.curdate;
    			var num1 = s.params.num1;
    			var num2 = s.params.num2;
    			var num3 = s.params.num3;
    			var num4 = s.params.num4;
    			
    			while(num1 <= num4) {
    				j('#progressbar').progressbar({ value: 5 });
    				j('#fech').html(curdat);
    				j('#mes').html('[1/3] Generando Poliza de Cargos...');
    				z = j.parseJSON(ajax('countPolCar', s.params.curdate, ''));
    				cur = z.cur;
    				//alert(z[1][1]);
    				valor = 5;
    				for(i=0;i<z.length;i++) {
    					expres = j.parseJSON(ajaxP('polCargosIns', {params: JSON.stringify(s.params), emp: z[i][0] }, '', ''));
    					perc = 100 / z[i][1];
    					for(e=0;e<z[i][1];e++) {
    						valor = valor + perc;
    						j('#progressbar').progressbar({ value: valor });
    						b = ajaxP('polCargos', {params: JSON.stringify(s.params), cur: e, emp: z[i][0], expression: expres.ex }, '', '');
    					}
    				}
    				j('#mes').html('Contabilizando...');
    				curdat = ajax('addDay', s.params.curdate, '');
    				s.params.curdate = curdat;
    				num1 = num1+1;
    				num2 = num2+1;
    			}
    		}
    	} else {
    		alert('La fecha final debe ser mayor a la fecha de inicio.');
    	}
    }
    Now you would normally say, well call async instead of sync, I know, but the problem is that if it's async I cannot assign it to a variable for one. I have to callback a function. If I do that how do I stay inside the same loop?

    As you can see I have to run a date loop to run through all days selected. Plus I also have to run through a mysql query loop, hence the returned Z data, there are 625 fields that are returned, for each field returned there are two companies and for all of them there are approx. 5 queries to run. So I have a loop that has to be run through.

    Any thoughts on how to do this? It works right now, it just doesn't update the DOM since it's frozen on a sync loop. Thanks for the help.
    "FORTRAN is not a language. It's a way of turning a multi-million dollar mainframe, into a $50 programmable scientific calculator."
    http://www.microfastcat.com -- FastCat Software, the fastest software on the NET!
    http://www.microthosting.com -- Free reseller web hosting, Hosting, VPS, FREE SMALL HOSTING!!!
    http://www.microtronix-tech.com -- Web design and programming

  • #2
    Senior Coder rnd me's Avatar
    Join Date
    Jun 2007
    Location
    Urbana
    Posts
    4,400
    Thanks
    11
    Thanked 595 Times in 575 Posts
    collect then report:

    Code:
    function square(n){return n*n;}
    r=[1,2,3,4,5];
    res=[];
    
    //sync solution:
    alert(r.map(square));
    
    
    //async solution:
    function done(){
      alert(res);
    }
    
    function addSolution(){
     var base=r[res.length]; 
     res.push(square(base));
     if(res.length>=r.length){ done(); }else{ setTimeout( addSolution, 500 );}
    }
    
    addSolution();
    my site (updated 13/9/26)
    BROWSER STATS [% share] (2014/9/03) IE7:0.1, IE8:4.6, IE11:9.1, IE9:3.1, IE10:3.0, FF:17.2, CH:46, SF:11.4, NON-MOUSE:38%

  • #3
    Regular Coder jfreak53's Avatar
    Join Date
    May 2004
    Location
    Guatemala
    Posts
    477
    Thanks
    19
    Thanked 10 Times in 10 Posts
    Yeah, but you can only have X amount of max async calls in a browser at one time, I would max way out with the 625 calls that it makes to the script. Plus there's no way to know if their are 625 records or less or more.
    "FORTRAN is not a language. It's a way of turning a multi-million dollar mainframe, into a $50 programmable scientific calculator."
    http://www.microfastcat.com -- FastCat Software, the fastest software on the NET!
    http://www.microthosting.com -- Free reseller web hosting, Hosting, VPS, FREE SMALL HOSTING!!!
    http://www.microtronix-tech.com -- Web design and programming

  • #4
    Senior Coder rnd me's Avatar
    Join Date
    Jun 2007
    Location
    Urbana
    Posts
    4,400
    Thanks
    11
    Thanked 595 Times in 575 Posts
    Quote Originally Posted by jfreak53 View Post
    Yeah, but you can only have X amount of max async calls in a browser at one time, I would max way out with the 625 calls that it makes to the script. Plus there's no way to know if their are 625 records or less or more.
    you could only do one call at a time, you never need more than a dozen hanging out there at once...

    you trigger the next call or batch of calls upon the arrival of a call.

    i'm not sure what your server spits out, but i faced a related challenge doing a CMS content dump, from a mysql db. i used LIMIT to ask for 100 rows at a time, range specified in the url. if the result set came back with 100 entries, i incremented the range by 100. if it had fewer than 100 rows, that set was complete and i invoked the collector callback to save the collated results to a file.


    to get started, i would break your code into more re-usable functions. get the logic out of the loop. once you get everything broken up, it will be much easier to start making it async.
    in particular,
    Code:
    while(num1 <= num4) {
    could be
    Code:
    function next(num1, num4) {
     if(num1>=num4){ return;}
     ...

    also, i would look at the sql, you should be able to combine a lot of those queries before they are served. use joins to create a wider table of data to avoid repeated calls.


    also, this line:
    Code:
    z = j.parseJSON(ajax('countPolCar', s.params.curdate, ''));
    can be moved to before the while loop (or function), which saves an IO every loop.
    my site (updated 13/9/26)
    BROWSER STATS [% share] (2014/9/03) IE7:0.1, IE8:4.6, IE11:9.1, IE9:3.1, IE10:3.0, FF:17.2, CH:46, SF:11.4, NON-MOUSE:38%

  • Users who have thanked rnd me for this post:

    jfreak53 (12-14-2011)

  • #5
    Regular Coder jfreak53's Avatar
    Join Date
    May 2004
    Location
    Guatemala
    Posts
    477
    Thanks
    19
    Thanked 10 Times in 10 Posts
    Hmm, ok now that makes more sense. Thanks, let me check into that tomorrow then. Thanks for the help.
    "FORTRAN is not a language. It's a way of turning a multi-million dollar mainframe, into a $50 programmable scientific calculator."
    http://www.microfastcat.com -- FastCat Software, the fastest software on the NET!
    http://www.microthosting.com -- Free reseller web hosting, Hosting, VPS, FREE SMALL HOSTING!!!
    http://www.microtronix-tech.com -- Web design and programming

  • #6
    Regular Coder jfreak53's Avatar
    Join Date
    May 2004
    Location
    Guatemala
    Posts
    477
    Thanks
    19
    Thanked 10 Times in 10 Posts
    WOW, worked like a charm! Thanks for the advise, now I just gotta finish all the other queries ha ha ha
    "FORTRAN is not a language. It's a way of turning a multi-million dollar mainframe, into a $50 programmable scientific calculator."
    http://www.microfastcat.com -- FastCat Software, the fastest software on the NET!
    http://www.microthosting.com -- Free reseller web hosting, Hosting, VPS, FREE SMALL HOSTING!!!
    http://www.microtronix-tech.com -- Web design and programming


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •