[dancer-users] 504 Gateway Timeout Error while processing big data files

[Tushar Dave]

Hello,

I am working on my Dancer Project (using Perl 5.20.0, apache2, RHEL 6 and
Dancer) in which I am providing a web form to user. When user submits the
form with appropriate values, a Q-Q plot will be generated by R code on
backend. Once the plot is available on server I am sending user an email
with the link of generated plot.

My code works completely fine with smaller data file (e.g. 100KB – 10 MB)
upload. But, if user uploads the large data files (e.g. > 30MB), the upload
works fine but processing of data and generating of plot will take more
time which caused 504 Gateway Time Out Error. I can use below options to
resolve this.

(1) Change Time Out time in apache configuration file
(2) Submit a job through batch queuing system such as SGE
(3) Use Proc::Simple or Proc::Background to run the process in background
and keep checking whether process is running in background or not.

I am using the 3rd option. What I am trying to do is I am using
Proc::Simple to run Perl-R-code (i.e. listed in below list as 2) in
background. Then I am using poll() method of Proc::Simple to check whether
the process is running or not. If the process is running, render a template
that shows user that process is still running. When process is finished,
render another template with generated Q-Q plot. Below are my codes:

(1) Dancer code under lib/ ---- http://pastebin.com/QGCFtq2X
<http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FQGCFtq2X&h=rAQEet8kg&enc=AZPs41qoqJIPTpi-4sL7ICzxgTszmdLN8joWBwLuzcrfK9vRcRfNFMBkMwQ469-n_nstqwOwtAbH-nmvLQ2wpy562EF9ef2zhBcdAmH-VMVJ0hxvwzRz0I7PTKh3UXO8cLkyOO0FHwL87zQbk5K3hOlh&s=1>
(2) Perl + R code that generate Q-Q plot ----http://pastebin.com/U5hg45dQ
(3) qqPlot.tt ---- http://pastebin.com/u30XArVr
(4) processing.tt ---- http://pastebin.com/RSH1kptp
<http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FRSH1kptp&h=RAQE1lHBj&enc=AZNQgweLY1hC15HguQHmiCGzFR4PmjOnNATBVnN1R6YQAB5GTbq8HVyUgPioxTRgWi5Ixb2d4HXAkvB9XC8336gnmRWA-ARYheUbQ0LrUZRyaQI_lpMVDD_D9ZL-F-79X6a4ZlLInL6O6QOknwU9RBpX&s=1>

The problem I am facing right now that my Dancer code (i.e. listed as 1
above) execute while loop on line 77 of (1) but the template “processing.tt”
is not coming up on browser. I don’t know what am I doing wrong?

Can someone help me to figure out my error? Or please suggest me any other
way to resolve Gateway Time Out Error.

Thanks for any help.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.preshweb.co.uk/pipermail/dancer-users/attachments/20141013/f89da308/attachment.html>