Discussion:
Par not working on Net::FTP::Recursive Script
(too old to reply)
Mike Flannigan
2015-08-02 17:51:52 UTC
Permalink
I have a script that runs fine in Perl64, but when I do

pp -o uploadfiles.exe inputfile.pl

and try to run uploadfiles.exe it opens a prompt
box and tries to connect to a HostGator account,
but after a few seconds gives
"login authentication failed".

The script uses this:

use strict;
use warnings;
use File::Find;
use Net::FTP::Recursive;
$| = 1; #Autoflush STDOUT


Anybody have any ideas on why this is not working?
This is Activestate Perl 5.16.3 built for
MSWin32-x64.


Mike
Roderich Schupp
2015-08-02 19:03:28 UTC
Permalink
Not enough information. For instance, how does it "open a prompt box"?
Try cutting the script down to a minimal example that exhibits the problem.

Cheers, Roderich
Mike Flannigan
2015-08-02 23:09:23 UTC
Permalink
Post by Roderich Schupp
Not enough information. For instance, how does it "open a prompt box"?
Try cutting the script down to a minimal example that exhibits the problem.
Cheers, Roderich
In Windows when you double click on a par exe file it opens
a command prompt box to show the results of the script.

I guess a minimalist script would be as follow:

use strict;
use warnings;
use Net::FTP::Recursive;

my $ip="142.146.2.103";
my $username="user";
my $pass="pass";

my $ftp = Net::FTP::Recursive->new($ip, Passive => 1, Debug => 0);

$ftp->login($username, $pass) or die $ftp->message;

print "All done.\n\a";

__END__


Interestingly, this seems to work in the par exe created
from this file.


OK, I haven't figured it out 100% yet, but it appears to have
something to do with the newline at the end of each of these
lines:

__DATA__
BkmTNBOhAQjsn3YUJWA9cw2JDp8lcw
Jx7ecihoLGsWidVUgRDl5CI38mpLbF
23X7p1HDSwXgGg4z8xSNshVBQScLCB
MkNyVS4Dnkx55TsW1VfCVm7niOU777

In my Windows Perl environment when I count
22 characters forward, I get the character I expect
to get. In the par environment, when I count
22 characters forward I get the character at position 21
instead (if it goes past the end of one of the lines). Perhaps
this has something to do with a newline being interpreted as
\n\r or \n or \cJ or whatever.

I'm probably going to have an ugly work-around for this. Or
perhaps I can come up with a more elegant work-around when
using par. We will see.

Thanks for your help.


Mike
Roderich Schupp
2015-08-03 14:44:39 UTC
Permalink
Post by Mike Flannigan
OK, I haven't figured it out 100% yet, but it appears to have
something to do with the newline at the end of each of these
__DATA__
BkmTNBOhAQjsn3YUJWA9cw2JDp8lcw
Good observation! I packed the following script (and checked that it has
Windows CRLF line endings)

--- snip ---
use strict;
use warnings;
use Data::Dumper;
$Data::Dumper::Useqq = 1;
my @data = <DATA>;
print Dumper(\@data);
__DATA__
fooy
bar
quux
--- snip ---

$ perl data.pl
$VAR1 = [
"foo\n",
"bar\n",
"quux\n"
];

but when I pack it

$ pp -o data.exe data.pl
$ .\data.exe
$VAR1 = [
"foo\r\n",
"bar\r\n",
"quux\r\n"
];

I checked the script as packed into data.exe and it has its CRLF endings
intact.
But the hack that's used to actually run it (i.e. PAR::_run_member) uses a
filehandle opened in binmode.
Looks like the implicit DATA filehandle inherits this setting somehow.

I'll have to think some more whether changing PAR::_run_member might have
side effects.

Cheers, Roderich
Mike Flannigan
2015-08-03 22:33:26 UTC
Permalink
Post by Roderich Schupp
Good observation! I packed the following script (and checked that it
has Windows CRLF line endings)
--- snip ---
use strict;
use warnings;
use Data::Dumper;
$Data::Dumper::Useqq = 1;
__DATA__
fooy
bar
quux
--- snip ---
$ perl data.pl <http://data.pl>
$VAR1 = [
"foo\n",
"bar\n",
"quux\n"
];
but when I pack it
$ pp -o data.exe data.pl <http://data.pl>
$ .\data.exe
$VAR1 = [
"foo\r\n",
"bar\r\n",
"quux\r\n"
];
I checked the script as packed into data.exe and it has its CRLF
endings intact.
But the hack that's used to actually run it (i.e. PAR::_run_member)
uses a filehandle opened in binmode.
Looks like the implicit DATA filehandle inherits this setting somehow.
I'll have to think some more whether changing PAR::_run_member might
have side effects.
Cheers, Roderich
Thanks for confirming that. Chomp certainly doesn't fix
the problem. Oddly I couldn't get the variable I wanted
no matter what I did! I always got the variable in front of
or behind the one I wanted, even though I only moved the
array index by one.

I finally gave up and removed all the newlines. I just
put all the data on one line now.

It's a bit frustrating, but I have my fix.
I'm surprised I was the one to point this out.
Must be very few of us Windows par users.


Mike
Shawn Laffan
2015-08-03 23:44:35 UTC
Permalink
Post by Mike Flannigan
Post by Roderich Schupp
Good observation! I packed the following script (and checked that it
has Windows CRLF line endings)
--- snip ---
use strict;
use warnings;
use Data::Dumper;
$Data::Dumper::Useqq = 1;
__DATA__
fooy
bar
quux
--- snip ---
$ perl data.pl <http://data.pl>
$VAR1 = [
"foo\n",
"bar\n",
"quux\n"
];
but when I pack it
$ pp -o data.exe data.pl <http://data.pl>
$ .\data.exe
$VAR1 = [
"foo\r\n",
"bar\r\n",
"quux\r\n"
];
I checked the script as packed into data.exe and it has its CRLF
endings intact.
But the hack that's used to actually run it (i.e. PAR::_run_member)
uses a filehandle opened in binmode.
Looks like the implicit DATA filehandle inherits this setting somehow.
I'll have to think some more whether changing PAR::_run_member might
have side effects.
Cheers, Roderich
Thanks for confirming that. Chomp certainly doesn't fix
the problem. Oddly I couldn't get the variable I wanted
no matter what I did! I always got the variable in front of
or behind the one I wanted, even though I only moved the
array index by one.
I finally gave up and removed all the newlines. I just
put all the data on one line now.
It's a bit frustrating, but I have my fix.
I'm surprised I was the one to point this out.
Must be very few of us Windows par users.
Mike
This is a more general problem than PAR.

I've had similar issues with input files when I was alternating between
cygwin and normal windows in the distant past. Data::Section::Simple
also does the same thing, in that it does not translate crlf endings
when loading from a data block on linux. I expect there are many other
ways of hitting this issue.

The solution I use is to remove the crlf line endings in my code, e.g.:

s/[\r\n]+$// for @data;

or, more verbosely but more clearly:

for my $item (@data) {
$item =~ s/[\r\n]+$//;
}

Regards,
Shawn.
Roderich Schupp
2015-08-04 14:03:00 UTC
Permalink
Post by Mike Flannigan
Thanks for confirming that. Chomp certainly doesn't fix
the problem.
chomp won't help, in my example it will only get you from

$VAR1 = [
"foo\r\n",
"bar\r\n",
"quux\r\n"
];

to

$VAR1 = [
"foo\r",
"bar\r",
"quux\r"
];

By default, chomp doesn't strip CR LF, as $/ is "\n" even on Windows.
Reading a file on Windows in non-binmode will convert CR LF to "\n".

It's a bit frustrating, but I have my fix.
I'm surprised I was the one to point this out.
Must be very few of us Windows par users.


My guess would be that the majority of PAR users is on Windows, but using
__DATA__ is rare.
Or at least using the data in a way that is susceptible to CRLF v LF
corruption.

Cheers, Roderich
Mike Flannigan
2015-08-08 12:12:27 UTC
Permalink
Post by Shawn Laffan
This is a more general problem than PAR.
I've had similar issues with input files when I was alternating
between cygwin and normal windows in the distant past.
Data::Section::Simple also does the same thing, in that it does not
translate crlf endings when loading from a data block on linux. I
expect there are many other ways of hitting this issue.
$item =~ s/[\r\n]+$//;
}
Regards,
Shawn.
Thanks for that fix. I was too lazy to carry this forward
until I saw your e-mail. Of course it worked. The script
works in both PAR and regular Perl when putting that regex
in there.

Thanks to everyone for all the help.


For my records I have summarized most of the e-mails on this
subject below.


Mike




------------------------------------------------------------------------



Subject:
Par not working on Net::FTP::Recursive Script
From:
Mike Flannigan <***@att.net>
Date:
8/2/2015 12:51 PM

To:
***@perl.org



I have a script that runs fine in Perl64, but when I do

pp -o uploadfiles.exe inputfile.pl

and try to run uploadfiles.exe it opens a prompt
box and tries to connect to a HostGator account,
but after a few seconds gives
"login authentication failed".

The script uses this:

use strict;
use warnings;
use File::Find;
use Net::FTP::Recursive;
$| = 1; #Autoflush STDOUT


Anybody have any ideas on why this is not working?
This is Activestate Perl 5.16.3 built for
MSWin32-x64.


Mike

------------------------------------------------------------------------



Subject:
Re: Par not working on Net::FTP::Recursive Script
From:
Roderich Schupp <***@gmail.com>
Date:
8/2/2015 2:03 PM

To:
Mike Flannigan <***@att.net>
CC:
"***@perl.org" <***@perl.org>


On Sun, Aug 2, 2015 at 7:51 PM, Mike Flannigan <***@att.net
<mailto:***@att.net>> wrote:

The script uses this:


Not enough information. For instance, how does it "open a prompt box"?
Try cutting the script down to a minimal example that exhibits the problem.

Cheers, Roderich


------------------------------------------------------------------------



Subject:
Re: Par not working on Net::FTP::Recursive Script
From:
Mike Flannigan <***@att.net>
Date:
8/2/2015 6:09 PM

To:
Roderich Schupp <***@gmail.com>
CC:
"***@perl.org" <***@perl.org>




In Windows when you double click on a par exe file it opens
a command prompt box to show the results of the script.

I guess a minimalist script would be as follow:

use strict;
use warnings;
use Net::FTP::Recursive;

my $ip="142.146.2.103";
my $username="user";
my $pass="pass";

my $ftp = Net::FTP::Recursive->new($ip, Passive => 1, Debug => 0);

$ftp->login($username, $pass) or die $ftp->message;

print "All done.\n\a";

__END__


Interestingly, this seems to work in the par exe created
from this file.


OK, I haven't figured it out 100% yet, but it appears to have
something to do with the newline at the end of each of these
lines:

__DATA__
BkmTNBOhAQjsn3YUJWA9cw2JDp8lcw
Jx7ecihoLGsWidVUgRDl5CI38mpLbF
23X7p1HDSwXgGg4z8xSNshVBQScLCB
MkNyVS4Dnkx55TsW1VfCVm7niOU777

In my Windows Perl environment when I count
22 characters forward, I get the character I expect
to get. In the par environment, when I count
22 characters forward I get the character at position 21
instead (if it goes past the end of one of the lines). Perhaps
this has something to do with a newline being interpreted as
\n\r or \n or \cJ or whatever.

I'm probably going to have an ugly work-around for this. Or
perhaps I can come up with a more elegant work-around when
using par. We will see.

Thanks for your help.


Mike


------------------------------------------------------------------------



Subject:
Re: Par not working on Net::FTP::Recursive Script
From:
Roderich Schupp <***@gmail.com>
Date:
8/3/2015 9:44 AM

To:
Mike Flannigan <***@att.net>
CC:
"***@perl.org" <***@perl.org>


On Mon, Aug 3, 2015 at 1:09 AM, Mike Flannigan <***@att.net
<mailto:***@att.net>> wrote:

OK, I haven't figured it out 100% yet, but it appears to have
something to do with the newline at the end of each of these
lines:

__DATA__
BkmTNBOhAQjsn3YUJWA9cw2JDp8lcw


Good observation! I packed the following script (and checked that it has
Windows CRLF line endings)

--- snip ---
use strict;
use warnings;
use Data::Dumper;
$Data::Dumper::Useqq = 1;
my @data = <DATA>;
print Dumper(\@data);
__DATA__
fooy
bar
quux
--- snip ---

$ perl data.pl <http://data.pl>
$VAR1 = [
"foo\n",
"bar\n",
"quux\n"
];

but when I pack it

$ pp -o data.exe data.pl <http://data.pl>
$ .\data.exe
$VAR1 = [
"foo\r\n",
"bar\r\n",
"quux\r\n"
];

I checked the script as packed into data.exe and it has its CRLF endings
intact.
But the hack that's used to actually run it (i.e. PAR::_run_member) uses
a filehandle opened in binmode.
Looks like the implicit DATA filehandle inherits this setting somehow.

I'll have to think some more whether changing PAR::_run_member might
have side effects.

Cheers, Roderich



------------------------------------------------------------------------


Subject:
Re: Par not working on Net::FTP::Recursive Script
From:
Mike Flannigan <***@att.net>
Date:
8/3/2015 5:33 PM

To:
Roderich Schupp <***@gmail.com>
CC:
"***@perl.org" <***@perl.org>




Thanks for confirming that. Chomp certainly doesn't fix
the problem. Oddly I couldn't get the variable I wanted
no matter what I did! I always got the variable in front of
or behind the one I wanted, even though I only moved the
array index by one.

I finally gave up and removed all the newlines. I just
put all the data on one line now.

It's a bit frustrating, but I have my fix.
I'm surprised I was the one to point this out.
Must be very few of us Windows par users.


Mike



------------------------------------------------------------------------


Subject:
Re: Par not working on Net::FTP::Recursive Script
From:
Roderich Schupp <***@gmail.com>
Date:
8/4/2015 9:03 AM

To:
Mike Flannigan <***@att.net>
CC:
"***@perl.org" <***@perl.org>


On Tue, Aug 4, 2015 at 12:33 AM, Mike Flannigan <***@att.net
<mailto:***@att.net>> wrote:

Thanks for confirming that. Chomp certainly doesn't fix
the problem.


chomp won't help, in my example it will only get you from

$VAR1 = [
"foo\r\n",
"bar\r\n",
"quux\r\n"
];

to

$VAR1 = [
"foo\r",
"bar\r",
"quux\r"
];

By default, chomp doesn't strip CR LF, as $/ is "\n" even on Windows.
Reading a file on Windows in non-binmode will convert CR LF to "\n".



My guess would be that the majority of PAR users is on Windows, but
using __DATA__ is rare.
Or at least using the data in a way that is susceptible to CRLF v LF
corruption.

Cheers, Roderich


------------------------------------------------------------------------




Subject:
Re: Par not working on Net::FTP::Recursive Script
From:
Shawn Laffan <***@unsw.edu.au>
Date:
8/3/2015 6:44 PM

To:
<***@perl.org>




On 4/08/2015 8:33, Mike Flannigan wrote:


This is a more general problem than PAR.

I've had similar issues with input files when I was alternating between
cygwin and normal windows in the distant past. Data::Section::Simple
also does the same thing, in that it does not translate crlf endings
when loading from a data block on linux. I expect there are many other
ways of hitting this issue.

The solution I use is to remove the crlf line endings in my code, e.g.:

s/[\r\n]+$// for @data;

or, more verbosely but more clearly:

for my $item (@data) {
$item =~ s/[\r\n]+$//;
}

Regards,
Shawn.
Shawn Laffan
2015-08-08 12:42:21 UTC
Permalink
No worries Mike.

After a bit more thought, the whole process might be simplified by
adding the :crlf layer to the DATA handle in your code. This is set by
default on windows, but not on linux.

e.g.:

binmode DATA, ':crlf';

while (my $line = <DATA>) {
do_stuff($line);
}


A few more details are at http://perldoc.perl.org/PerlIO.html#DESCRIPTION

Regards,
Shawn.
Post by Mike Flannigan
Post by Shawn Laffan
This is a more general problem than PAR.
I've had similar issues with input files when I was alternating
between cygwin and normal windows in the distant past.
Data::Section::Simple also does the same thing, in that it does not
translate crlf endings when loading from a data block on linux. I
expect there are many other ways of hitting this issue.
$item =~ s/[\r\n]+$//;
}
Regards,
Shawn.
Thanks for that fix. I was too lazy to carry this forward
until I saw your e-mail. Of course it worked. The script
works in both PAR and regular Perl when putting that regex
in there.
Thanks to everyone for all the help.
For my records I have summarized most of the e-mails on this
subject below.
Mike
------------------------------------------------------------------------
Par not working on Net::FTP::Recursive Script
8/2/2015 12:51 PM
I have a script that runs fine in Perl64, but when I do
pp -o uploadfiles.exe inputfile.pl
and try to run uploadfiles.exe it opens a prompt
box and tries to connect to a HostGator account,
but after a few seconds gives
"login authentication failed".
use strict;
use warnings;
use File::Find;
use Net::FTP::Recursive;
$| = 1; #Autoflush STDOUT
Anybody have any ideas on why this is not working?
This is Activestate Perl 5.16.3 built for
MSWin32-x64.
Mike
------------------------------------------------------------------------
Re: Par not working on Net::FTP::Recursive Script
8/2/2015 2:03 PM
Not enough information. For instance, how does it "open a prompt box"?
Try cutting the script down to a minimal example that exhibits the problem.
Cheers, Roderich
------------------------------------------------------------------------
Re: Par not working on Net::FTP::Recursive Script
8/2/2015 6:09 PM
In Windows when you double click on a par exe file it opens
a command prompt box to show the results of the script.
use strict;
use warnings;
use Net::FTP::Recursive;
my $ip="142.146.2.103";
my $username="user";
my $pass="pass";
my $ftp = Net::FTP::Recursive->new($ip, Passive => 1, Debug => 0);
$ftp->login($username, $pass) or die $ftp->message;
print "All done.\n\a";
__END__
Interestingly, this seems to work in the par exe created
from this file.
OK, I haven't figured it out 100% yet, but it appears to have
something to do with the newline at the end of each of these
__DATA__
BkmTNBOhAQjsn3YUJWA9cw2JDp8lcw
Jx7ecihoLGsWidVUgRDl5CI38mpLbF
23X7p1HDSwXgGg4z8xSNshVBQScLCB
MkNyVS4Dnkx55TsW1VfCVm7niOU777
In my Windows Perl environment when I count
22 characters forward, I get the character I expect
to get. In the par environment, when I count
22 characters forward I get the character at position 21
instead (if it goes past the end of one of the lines). Perhaps
this has something to do with a newline being interpreted as
\n\r or \n or \cJ or whatever.
I'm probably going to have an ugly work-around for this. Or
perhaps I can come up with a more elegant work-around when
using par. We will see.
Thanks for your help.
Mike
------------------------------------------------------------------------
Re: Par not working on Net::FTP::Recursive Script
8/3/2015 9:44 AM
OK, I haven't figured it out 100% yet, but it appears to have
something to do with the newline at the end of each of these
__DATA__
BkmTNBOhAQjsn3YUJWA9cw2JDp8lcw
Good observation! I packed the following script (and checked that it
has Windows CRLF line endings)
--- snip ---
use strict;
use warnings;
use Data::Dumper;
$Data::Dumper::Useqq = 1;
__DATA__
fooy
bar
quux
--- snip ---
$ perl data.pl <http://data.pl>
$VAR1 = [
"foo\n",
"bar\n",
"quux\n"
];
but when I pack it
$ pp -o data.exe data.pl <http://data.pl>
$ .\data.exe
$VAR1 = [
"foo\r\n",
"bar\r\n",
"quux\r\n"
];
I checked the script as packed into data.exe and it has its CRLF
endings intact.
But the hack that's used to actually run it (i.e. PAR::_run_member)
uses a filehandle opened in binmode.
Looks like the implicit DATA filehandle inherits this setting somehow.
I'll have to think some more whether changing PAR::_run_member might
have side effects.
Cheers, Roderich
------------------------------------------------------------------------
Re: Par not working on Net::FTP::Recursive Script
8/3/2015 5:33 PM
Thanks for confirming that. Chomp certainly doesn't fix
the problem. Oddly I couldn't get the variable I wanted
no matter what I did! I always got the variable in front of
or behind the one I wanted, even though I only moved the
array index by one.
I finally gave up and removed all the newlines. I just
put all the data on one line now.
It's a bit frustrating, but I have my fix.
I'm surprised I was the one to point this out.
Must be very few of us Windows par users.
Mike
------------------------------------------------------------------------
Re: Par not working on Net::FTP::Recursive Script
8/4/2015 9:03 AM
Thanks for confirming that. Chomp certainly doesn't fix
the problem.
chomp won't help, in my example it will only get you from
$VAR1 = [
"foo\r\n",
"bar\r\n",
"quux\r\n"
];
to
$VAR1 = [
"foo\r",
"bar\r",
"quux\r"
];
By default, chomp doesn't strip CR LF, as $/ is "\n" even on Windows.
Reading a file on Windows in non-binmode will convert CR LF to "\n".
My guess would be that the majority of PAR users is on Windows, but
using __DATA__ is rare.
Or at least using the data in a way that is susceptible to CRLF v LF
corruption.
Cheers, Roderich
------------------------------------------------------------------------
Re: Par not working on Net::FTP::Recursive Script
8/3/2015 6:44 PM
This is a more general problem than PAR.
I've had similar issues with input files when I was alternating
between cygwin and normal windows in the distant past.
Data::Section::Simple also does the same thing, in that it does not
translate crlf endings when loading from a data block on linux. I
expect there are many other ways of hitting this issue.
$item =~ s/[\r\n]+$//;
}
Regards,
Shawn.
Roderich Schupp
2015-08-08 13:52:09 UTC
Permalink
After a bit more thought, the whole process might be simplified by adding
the :crlf layer to the DATA handle in your code. This is set by default on
windows, but not on linux.
That should also help when you move files between Windows and *nix without
paying attention to line endings, because Perl itself (when parsing the
source) doesn't care.

Cheers, Roderich

Loading...