[dancer-users] 504 Gateway Timeout Error while processing big data files
Tushar Dave
tushardave26 at gmail.com
Tue Oct 14 21:20:01 BST 2014
Hello Hughes,
Thanks for your solution. Can you elaborate the process by using my code? I
am newbie in web development and never worked on AJAX. So I am having
little bit hard time to connect front-end and back-end pieces. I hope this
make sense. Thanks again for your help.
Best,
Tushar
On Tue, Oct 14, 2014 at 7:00 AM, <dancer-users-request at dancer.pm> wrote:
> Send dancer-users mailing list submissions to
> dancer-users at dancer.pm
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.preshweb.co.uk/mailman/listinfo/dancer-users
> or, via email, send a message with subject or body 'help' to
> dancer-users-request at dancer.pm
>
> You can reach the person managing the list at
> dancer-users-owner at dancer.pm
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of dancer-users digest..."
>
>
> Today's Topics:
>
> 1. 504 Gateway Timeout Error while processing big data files
> (Tushar Dave)
> 2. Dancer2 0.152000 on its way to CPAN (Sawyer X)
> 3. Re: 504 Gateway Timeout Error while processing big data files
> (Hugues)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 13 Oct 2014 09:45:12 -0400
> From: Tushar Dave <tushardave26 at gmail.com>
> To: dancer-users at dancer.pm
> Subject: [dancer-users] 504 Gateway Timeout Error while processing big
> data files
> Message-ID:
> <CA+8TAD=
> AV+-bPirEK_HtJKV6MzPSsicU+hHWES-ZWvDu30AQqg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hello,
>
> I am working on my Dancer Project (using Perl 5.20.0, apache2, RHEL 6 and
> Dancer) in which I am providing a web form to user. When user submits the
> form with appropriate values, a Q-Q plot will be generated by R code on
> backend. Once the plot is available on server I am sending user an email
> with the link of generated plot.
>
> My code works completely fine with smaller data file (e.g. 100KB ? 10 MB)
> upload. But, if user uploads the large data files (e.g. > 30MB), the upload
> works fine but processing of data and generating of plot will take more
> time which caused 504 Gateway Time Out Error. I can use below options to
> resolve this.
>
> (1) Change Time Out time in apache configuration file
> (2) Submit a job through batch queuing system such as SGE
> (3) Use Proc::Simple or Proc::Background to run the process in background
> and keep checking whether process is running in background or not.
>
> I am using the 3rd option. What I am trying to do is I am using
> Proc::Simple to run Perl-R-code (i.e. listed in below list as 2) in
> background. Then I am using poll() method of Proc::Simple to check whether
> the process is running or not. If the process is running, render a template
> that shows user that process is still running. When process is finished,
> render another template with generated Q-Q plot. Below are my codes:
>
> (1) Dancer code under lib/ ---- http://pastebin.com/QGCFtq2X
> <
> http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FQGCFtq2X&h=rAQEet8kg&enc=AZPs41qoqJIPTpi-4sL7ICzxgTszmdLN8joWBwLuzcrfK9vRcRfNFMBkMwQ469-n_nstqwOwtAbH-nmvLQ2wpy562EF9ef2zhBcdAmH-VMVJ0hxvwzRz0I7PTKh3UXO8cLkyOO0FHwL87zQbk5K3hOlh&s=1
> >
> (2) Perl + R code that generate Q-Q plot ----http://pastebin.com/U5hg45dQ
> (3) qqPlot.tt ---- http://pastebin.com/u30XArVr
> (4) processing.tt ---- http://pastebin.com/RSH1kptp
> <
> http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FRSH1kptp&h=RAQE1lHBj&enc=AZNQgweLY1hC15HguQHmiCGzFR4PmjOnNATBVnN1R6YQAB5GTbq8HVyUgPioxTRgWi5Ixb2d4HXAkvB9XC8336gnmRWA-ARYheUbQ0LrUZRyaQI_lpMVDD_D9ZL-F-79X6a4ZlLInL6O6QOknwU9RBpX&s=1
> >
>
> The problem I am facing right now that my Dancer code (i.e. listed as 1
> above) execute while loop on line 77 of (1) but the template ?
> processing.tt?
> is not coming up on browser. I don?t know what am I doing wrong?
>
> Can someone help me to figure out my error? Or please suggest me any other
> way to resolve Gateway Time Out Error.
>
> Thanks for any help.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.preshweb.co.uk/pipermail/dancer-users/attachments/20141013/f89da308/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Tue, 14 Oct 2014 04:49:05 +0200
> From: Sawyer X <xsawyerx at gmail.com>
> To: Perl Dancer users mailing list <dancer-users at dancer.pm>
> Subject: [dancer-users] Dancer2 0.152000 on its way to CPAN
> Message-ID:
> <CAMvkq_Tvx07eM=TK+ww8mung2UpiAX89ayWzxof5=
> zTiOcxe-w at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi everyone!
>
> We've done quite a few big things in Dancer2 and I'd like to share them. I
> haven't sent an email about 0.151000 - released last week - so I'm going to
> share some information about it too.
>
> These two versions provide a few important changes:
> * Introducing our policy document.
> * Dancer2 Apps (Dancer2::Core::App) now build proper PSGI applications -
> *each
> on its own*. This allows you to create much better PSGI apps, which will
> interact much better. This will also disallow things we do not recommend,
> such as "pass" and "forward" between Dancer Apps. The original `psgi_app`
> keyword (and Dancer2 method) still works, just as before, providing the old
> behavior.
> * We added a migration document, for those moving from Dancer to Dancer2,
> thanks to Snigdha Dagar.
> * There is a big "chat with us" banner on MetaCPAN.org, to help newbies
> join the IRC channel.
> * New 'content' keyword to set content in a declarative manner.
>
> The first two are the biggest ones. I will email the list about both of
> those separately.
>
> I would like to congratulate and thank all those who worked on this
> release: Andy Beverley, D?vid Kov?cs, DavsX, Paul
> Cochrane, simbabque, Lindsey Beesley, Snigdha Dagar, Russell Jenkins, and
> yours truly, Sawyer X.
>
> Great work, everyone!
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.preshweb.co.uk/pipermail/dancer-users/attachments/20141014/a7fae5c9/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Tue, 14 Oct 2014 11:02:00 +0200
> From: Hugues <hugues at max4mail.com>
> To: dancer-users at dancer.pm
> Subject: Re: [dancer-users] 504 Gateway Timeout Error while processing
> big data files
> Message-ID: <543CE688.4000701 at max4mail.com>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>
> Hello Dave
> for my point of view , the problem come from file
> http://pastebin.com/QGCFtq2X
> <
> http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FQGCFtq2X&h=rAQEet8kg&enc=AZPs41qoqJIPTpi-4sL7ICzxgTszmdLN8joWBwLuzcrfK9vRcRfNFMBkMwQ469-n_nstqwOwtAbH-nmvLQ2wpy562EF9ef2zhBcdAmH-VMVJ0hxvwzRz0I7PTKh3UXO8cLkyOO0FHwL87zQbk5K3hOlh&s=1
> >
> line 77 to 80
>
> while ($process->poll()) {
> template 'processing.tt';
> sleep 10;
> }
> first time it's work, but after not , every 10 sec you send
> processing.tt to your browser ( first time ,
> browser ask 1 anwers, not more , and that all )
>
> you need to use ajax
> 1) launch in background your process (and create PID to locate your
> process, https://metacpan.org/pod/Unix::PID )
>
> 2) add javascript in your brower to update result ( see example below )
>
> 3) add route
>
> ajax '/myProcesss' {
>
> If ( $PID ) {
> ... in course
> } else {
> ... process over
> }
>
> }
>
>
>
> see fun example
> http://advent.perldancer.org/2011/14
>
> bye
> Hugues
>
>
>
> Le 13/10/2014 15:45, Tushar Dave a ?crit :
> >
> > Hello,
> >
> > I am working on my Dancer Project (using Perl 5.20.0, apache2, RHEL 6
> > and Dancer) in which I am providing a web form to user. When user
> > submits the form with appropriate values, a Q-Q plot will be generated
> > by R code on backend. Once the plot is available on server I am
> > sending user an email with the link of generated plot.
> >
> > My code works completely fine with smaller data file (e.g. 100KB ? 10
> > MB) upload. But, if user uploads the large data files (e.g. > 30MB),
> > the upload works fine but processing of data and generating of plot
> > will take more time which caused 504 Gateway Time Out Error. I can use
> > below options to resolve this.
> >
> > (1) Change Time Out time in apache configuration file
> > (2) Submit a job through batch queuing system such as SGE
> > (3) Use Proc::Simple or Proc::Background to run the process in
> > background and keep checking whether process is running in background
> > or not.
> >
> > I am using the 3rd option. What I am trying to do is I am using
> > Proc::Simple to run Perl-R-code (i.e. listed in below list as 2) in
> > background. Then I am using poll() method of Proc::Simple to check
> > whether the process is running or not. If the process is running,
> > render a template that shows user that process is still running. When
> > process is finished, render another template with generated Q-Q plot.
> > Below are my codes:
> >
> > (1) Dancer code under lib/ ---- http://pastebin.com/QGCFtq2X
> > <
> http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FQGCFtq2X&h=rAQEet8kg&enc=AZPs41qoqJIPTpi-4sL7ICzxgTszmdLN8joWBwLuzcrfK9vRcRfNFMBkMwQ469-n_nstqwOwtAbH-nmvLQ2wpy562EF9ef2zhBcdAmH-VMVJ0hxvwzRz0I7PTKh3UXO8cLkyOO0FHwL87zQbk5K3hOlh&s=1
> >
> > (2) Perl + R code that generate Q-Q plot ----
> http://pastebin.com/U5hg45dQ
> > (3) qqPlot.tt ---- http://pastebin.com/u30XArVr
> > (4) processing.tt <http://processing.tt> ----
> > http://pastebin.com/RSH1kptp
> > <
> http://l.facebook.com/l.php?u=http%3A%2F%2Fpastebin.com%2FRSH1kptp&h=RAQE1lHBj&enc=AZNQgweLY1hC15HguQHmiCGzFR4PmjOnNATBVnN1R6YQAB5GTbq8HVyUgPioxTRgWi5Ixb2d4HXAkvB9XC8336gnmRWA-ARYheUbQ0LrUZRyaQI_lpMVDD_D9ZL-F-79X6a4ZlLInL6O6QOknwU9RBpX&s=1
> >
> >
> > The problem I am facing right now that my Dancer code (i.e. listed as
> > 1 above) execute while loop on line 77 of (1) but the template
> > ?processing.tt <http://processing.tt>? is not coming up on browser. I
> > don?t know what am I doing wrong?
> >
> > Can someone help me to figure out my error? Or please suggest me any
> > other way to resolve Gateway Time Out Error.
> >
> > Thanks for any help.
> >
> >
> >
> >
> >
> > _______________________________________________
> > dancer-users mailing list
> > dancer-users at dancer.pm
> > http://lists.preshweb.co.uk/mailman/listinfo/dancer-users
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.preshweb.co.uk/pipermail/dancer-users/attachments/20141014/fc54d1f3/attachment-0001.html
> >
>
> ------------------------------
>
> _______________________________________________
> dancer-users mailing list
> dancer-users at dancer.pm
> http://lists.preshweb.co.uk/mailman/listinfo/dancer-users
>
>
> End of dancer-users Digest, Vol 56, Issue 12
> ********************************************
>
--
Tushar Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.preshweb.co.uk/pipermail/dancer-users/attachments/20141014/97f3176c/attachment-0001.html>
More information about the dancer-users
mailing list