504 Gateway Timeout Error while processing big data files
Hello, I am working on my Dancer Project (using Perl 5.20.0, apache2, RHEL 6 and Dancer) in which I am providing a web form to user. When user submits the form with appropriate values, a Q-Q plot will be generated by R code on backend. Once the plot is available on server I am sending user an email with the link of generated plot. My code works completely fine with smaller data file (e.g. 100KB – 10 MB) upload. But, if user uploads the large data files (e.g. > 30MB), the upload works fine but processing of data and generating of plot will take more time which caused 504 Gateway Time Out Error. I can use below options to resolve this. (1) Change Time Out time in apache configuration file (2) Submit a job through batch queuing system such as SGE (3) Use Proc::Simple or Proc::Background to run the process in background and keep checking whether process is running in background or not. I am using the 3rd option. What I am trying to do is I am using Proc::Simple to run Perl-R-code (i.e. listed in below list as 2) in background. Then I am using poll() method of Proc::Simple to check whether the process is running or not. If the process is running, render a template that shows user that process is still running. When process is finished, render another template with generated Q-Q plot. Below are my codes: (1) Dancer code under lib/ ---- http://pastebin.com/QGCFtq2X <http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FQGCFtq2X&h=rAQEet8kg&enc=AZPs41qoqJIPTpi-4sL7ICzxgTszmdLN8joWBwLuzcrfK9vRcRfNFMBkMwQ469-n_nstqwOwtAbH-nmvLQ2wpy562EF9ef2zhBcdAmH-VMVJ0hxvwzRz0I7PTKh3UXO8cLkyOO0FHwL87zQbk5K3hOlh&s=1> (2) Perl + R code that generate Q-Q plot ----http://pastebin.com/U5hg45dQ (3) qqPlot.tt ---- http://pastebin.com/u30XArVr (4) processing.tt ---- http://pastebin.com/RSH1kptp <http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FRSH1kptp&h=RAQE1lHBj&enc=AZNQgweLY1hC15HguQHmiCGzFR4PmjOnNATBVnN1R6YQAB5GTbq8HVyUgPioxTRgWi5Ixb2d4HXAkvB9XC8336gnmRWA-ARYheUbQ0LrUZRyaQI_lpMVDD_D9ZL-F-79X6a4ZlLInL6O6QOknwU9RBpX&s=1> The problem I am facing right now that my Dancer code (i.e. listed as 1 above) execute while loop on line 77 of (1) but the template “processing.tt” is not coming up on browser. I don’t know what am I doing wrong? Can someone help me to figure out my error? Or please suggest me any other way to resolve Gateway Time Out Error. Thanks for any help.
Hello Dave for my point of view , the problem come from file http://pastebin.com/QGCFtq2X <http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FQGCFtq2X&h=rAQEet8kg&enc=AZPs41qoqJIPTpi-4sL7ICzxgTszmdLN8joWBwLuzcrfK9vRcRfNFMBkMwQ469-n_nstqwOwtAbH-nmvLQ2wpy562EF9ef2zhBcdAmH-VMVJ0hxvwzRz0I7PTKh3UXO8cLkyOO0FHwL87zQbk5K3hOlh&s=1> line 77 to 80 while ($process->poll()) { template 'processing.tt'; sleep 10; } first time it's work, but after not , every 10 sec you send processing.tt to your browser ( first time , browser ask 1 anwers, not more , and that all ) you need to use ajax 1) launch in background your process (and create PID to locate your process, https://metacpan.org/pod/Unix::PID ) 2) add javascript in your brower to update result ( see example below ) 3) add route ajax '/myProcesss' { If ( $PID ) { ... in course } else { ... process over } } see fun example http://advent.perldancer.org/2011/14 bye Hugues Le 13/10/2014 15:45, Tushar Dave a écrit :
Hello,
I am working on my Dancer Project (using Perl 5.20.0, apache2, RHEL 6 and Dancer) in which I am providing a web form to user. When user submits the form with appropriate values, a Q-Q plot will be generated by R code on backend. Once the plot is available on server I am sending user an email with the link of generated plot.
My code works completely fine with smaller data file (e.g. 100KB – 10 MB) upload. But, if user uploads the large data files (e.g. > 30MB), the upload works fine but processing of data and generating of plot will take more time which caused 504 Gateway Time Out Error. I can use below options to resolve this.
(1) Change Time Out time in apache configuration file (2) Submit a job through batch queuing system such as SGE (3) Use Proc::Simple or Proc::Background to run the process in background and keep checking whether process is running in background or not.
I am using the 3rd option. What I am trying to do is I am using Proc::Simple to run Perl-R-code (i.e. listed in below list as 2) in background. Then I am using poll() method of Proc::Simple to check whether the process is running or not. If the process is running, render a template that shows user that process is still running. When process is finished, render another template with generated Q-Q plot. Below are my codes:
(1) Dancer code under lib/ ---- http://pastebin.com/QGCFtq2X <http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FQGCFtq2X&h=rAQEet8kg&enc=AZPs41qoqJIPTpi-4sL7ICzxgTszmdLN8joWBwLuzcrfK9vRcRfNFMBkMwQ469-n_nstqwOwtAbH-nmvLQ2wpy562EF9ef2zhBcdAmH-VMVJ0hxvwzRz0I7PTKh3UXO8cLkyOO0FHwL87zQbk5K3hOlh&s=1> (2) Perl + R code that generate Q-Q plot ----http://pastebin.com/U5hg45dQ (3) qqPlot.tt ---- http://pastebin.com/u30XArVr (4) processing.tt <http://processing.tt> ---- http://pastebin.com/RSH1kptp <http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FRSH1kptp&h=RAQE1lHBj&enc=AZNQgweLY1hC15HguQHmiCGzFR4PmjOnNATBVnN1R6YQAB5GTbq8HVyUgPioxTRgWi5Ixb2d4HXAkvB9XC8336gnmRWA-ARYheUbQ0LrUZRyaQI_lpMVDD_D9ZL-F-79X6a4ZlLInL6O6QOknwU9RBpX&s=1>
The problem I am facing right now that my Dancer code (i.e. listed as 1 above) execute while loop on line 77 of (1) but the template “processing.tt <http://processing.tt>” is not coming up on browser. I don’t know what am I doing wrong?
Can someone help me to figure out my error? Or please suggest me any other way to resolve Gateway Time Out Error.
Thanks for any help.
_______________________________________________ dancer-users mailing list dancer-users@dancer.pm http://lists.preshweb.co.uk/mailman/listinfo/dancer-users
participants (2)
-
Hugues -
Tushar Dave