Discussion:
Adding local directories to @INC for a pp executable
(too old to reply)
(Harald Jörg)
2015-04-20 21:05:35 UTC
Permalink
Hello PARty,

I'm new to PAR and pp and am trying to use it to package a fairly large
application so that Windows users who don't have Perl installed can
simply double-click the application. I've managed to overcome some
pitfalls (missing DLLs, dynamic loading of modules which Scandeps can't
find), but there's still one thing I don't understand, and Google didn't
point me into an explanation so far.

What I'm trying to achieve: The Perl application can be extended with a
plugin mechanism. In plain Perl, this is done by dropping the modules
to special subdirectories, the application will pick them up. Now, I
want to let users do the same thing on top of the pp executable.

So I thought that I could just

use lib 'foo';

in my source, and then drop the plugins to directory 'foo'. To my
disappointment, this fails, the modules aren't found. Drats.
To my utter surprise, I got it working like this:

use lib 'foo';
use lib 'bar';

...and then dropping the plugins to 'foo'. Now the questions:

* is that intended / accidental behaviour?
* is there a better way to do what I want?

Here's a simple demo: 'foo' is added to @INC as intended, 'bar' isn't.
----------------------------------------------------------------------
$ cat local_includes.pl
use Data::Dumper;
use lib 'foo';
use lib 'bar';
print Dumper(\@INC);
$ pp local_includes.pl
$ ./a.out
$VAR1 = [
'foo',
sub { "DUMMY" },
'/tmp/par-68616a/cache-5bde0b3f6eab9c5f0be91ed26371a4bb8b7cbac1/inc/lib',
'/tmp/par-68616a/cache-5bde0b3f6eab9c5f0be91ed26371a4bb8b7cbac1/inc',
sub { "DUMMY" },
sub { "DUMMY" }
];
----------------------------------------------------------------------

--
Cheers,
haj
Roderich Schupp
2015-04-21 07:21:39 UTC
Permalink
On Mon, Apr 20, 2015 at 11:05 PM, Harald Jörg <***@arcor.de> wrote:

So I thought that I could just
Post by (Harald Jörg)
use lib 'foo';
in my source, and then drop the plugins to directory 'foo'. To my
Post by (Harald Jörg)
disappointment, this fails, the modules aren't found. Drats.
use lib 'foo';
use lib 'bar';
* is that intended / accidental behaviour?
That's definitely a bug, please submit a bug report to rt.cpan.org (in
queue PAR::Packer).

I'm able to reproduce this. On first examination, lib::import() is
definitely called twice
and does the right thing (ie. prepending 'foo' and 'bar' to @INC), but then
'foo' vanishes
from @INC. Might be some weird side effect of local'izing @INC.

I've managed to overcome some
Post by (Harald Jörg)
pitfalls (missing DLLs, dynamic loading of modules which Scandeps can't
find), but there's still one thing I don't understand, and Google didn't
point me into an explanation so far.
If these are public modules (ie. they are on CPAN), please
consider submitting bugs
for these as well (queue Module::ScanDeps).

Cheers, Roderich
Roderich Schupp
2015-04-22 16:32:18 UTC
Permalink
The DLLs are the notorious libeay32_.dll and ssleay32_.dll.
I guess zlib1_.dll, as used by
Archive::Tar, is of the same sort. Hmmm... I can --link them to the
executable, but after unpacking to tempdir they end up in a directory
where apparently Windows doesn't look for them.
pp -o ssleay.exe --link libeay32_.dll --link ssleay32_.dll -E "use
Net::SSLeay; say 'OK';

And this places these dlls into shlib/MSWin32-x86-multi-thread-64int in the
packed executable:

$ unzip -l ssleay.exe | grep -i dll
27136 09-15-2014 20:10 lib/auto/File/Glob/Glob.xs.dll
44544 09-15-2014 20:48 lib/auto/List/Util/Util.xs.dll
19456 09-15-2014 20:10 lib/auto/MIME/Base64/Base64.xs.dll
350208 09-15-2014 21:42 lib/auto/Net/SSLeay/SSLeay.xs.dll
36352 09-15-2014 20:34 lib/auto/Socket/Socket.xs.dll
1764352 07-28-2014 02:31
shlib/MSWin32-x86-multi-thread-64int/libeay32_.dll
394752 07-28-2014 02:31
shlib/MSWin32-x86-multi-thread-64int/ssleay32_.dll

And they will be extracted to the top level cache directory which is in the
PATH
when the executable is run.

I can provide a list of CPAN module dependencies which Module::ScanDeps
doesn't find to its queue. However, these dependencies are only
required if certain conditions are met. For applications where these
conditions don't apply, the PAR gets bloated without need.
Please do - PAR::Packer (and Module::ScanDeps) try to err on the generous
side.
For example, if there's the slightest whiff that a script might be using
unicode stuff,
the whole machinery will be included.

Cheers, Roderich
(Harald Jörg)
2015-04-22 20:04:27 UTC
Permalink
The DLLs are the notorious libeay32_.dll and ssleay32_.dll.
I guess zlib1_.dll, as used by Archive::Tar, is of the same
sort. Hmmm... I can --link them to the executable, but after
unpacking to tempdir they end up in a directory where apparently
Windows doesn't look for them.
pp -o ssleay.exe --link libeay32_.dll --link ssleay32_.dll -E "use
Net::SSLeay; say 'OK';
And this places these dlls into shlib/MSWin32-x86-multi-thread-64int
$ unzip -l ssleay.exe | grep -i dll
[...]
shlib/MSWin32-x86-multi-thread-64int/libeay32_.dll
shlib/MSWin32-x86-multi-thread-64int/ssleay32_.dll
Yes, that's what I had observed, too.
And they will be extracted to the top level cache directory which is
in the PATH when the executable is run.
Oh - I didn't know that. So I re-ran the whole build procedure (still
using PAR 1.008), and indeed - it works just fine, for SSL and Zlib!

Apparently I had made some mistake during the build processes. Probably
I re-built without the proper --link option, but still stared at the old
executable. Then, after seeing the "library not found" error, I drew
the wrong conclusions and decided to use the workaround of copying the
DLLs to the directory where the executable sits.

### My apologies: pp works as desired with DLLs! ###
I can provide a list of CPAN module dependencies which
Module::ScanDeps doesn't find to its queue. However, these
dependencies are only required if certain conditions are met. For
applications where these conditions don't apply, the PAR gets
bloated without need.
Please do - PAR::Packer (and Module::ScanDeps) try to err on the
generous side. For example, if there's the slightest whiff that a
script might be using unicode stuff, the whole machinery will be
included.
Ok then... I'll have a look what is required directly by the application
and what comes from CPAN modules. Plack for sure falls into the "whole
machinery" category :)

Thanks again for your support!
--
Cheers,
haj

Roderich Schupp
2015-04-22 16:43:22 UTC
Permalink
The DLLs are the notorious libeay32_.dll and ssleay32_.dll. I seem to
recall having found an article of yours saying this can't be solved, but
am unable to find the article right now.
Of course it can be solved, but not by me :)

A general solution for all major OS platforms (inclduigna diversity of tool
chains) is indeed hard work,
but if one restricts itself to, say, Windows with the MingW toolchain (esp.
the objdump program) [1],
this is actually grepping the output of "objdump -x ..." and searching
$PATH for the DLLs mentioned
there. (Of course you have to filter out system DLLs like kernel32.dll.)

For more inspiration, look up py2exe on pypi.python.org. It does the same
for Python
as PAR::Packer for Perl. It's Windows only, though. They have solved the
"external DLL" problem.
They have done real crazy shit, like including a DLL runtime loader that
load from a zip
_without_ extracting to the file system.

Cheers, Roderich



[1] always included with Strawberry Perl, and just a ppm install away with
ActiveState perl
(Harald Jörg)
2015-04-21 18:57:03 UTC
Permalink
Post by (Harald Jörg)
So I thought that I could just
use lib 'foo';
in my source, and then drop the plugins to directory 'foo'. To my
disappointment, this fails, the modules aren't found. Drats.
[...]
* is that intended / accidental behaviour?
That's definitely a bug, please submit a bug report to rt.cpan.org (in
queue PAR::Packer).
Thanks for the clarification! I've submitted a bug report.
Post by (Harald Jörg)
[...]
I've managed to overcome some pitfalls (missing DLLs, dynamic
loading of modules which Scandeps can't find), but there's still
one thing I don't understand, and Google didn't point me into an
explanation so far.
If these are public modules (ie. they are on CPAN), please consider
submitting bugs for these as well (queue Module::ScanDeps).
The DLLs are the notorious libeay32_.dll and ssleay32_.dll. I seem to
recall having found an article of yours saying this can't be solved, but
am unable to find the article right now. I guess zlib1_.dll, as used by
Archive::Tar, is of the same sort. Hmmm... I can --link them to the
executable, but after unpacking to tempdir they end up in a directory
where apparently Windows doesn't look for them. Would adding that
directory to $ENV{PATH} solve the issue? This would be a job of the
unpacker, right?

I can provide a list of CPAN module dependencies which Module::ScanDeps
doesn't find to its queue. However, these dependencies are only
required if certain conditions are met. For applications where these
conditions don't apply, the PAR gets bloated without need.

Actually, this problem is rather easy to workaround: I added a module
which just uses the undetected dependencies. I need this anyway, for
dynamic loading by the application itself. So if I submit a bug report
and it gets rejected, I'm fine with it - at least we leave a trace for
Googlers who encounter the same issues.

Thanks again for your support!
--
Cheers,
haj
BTW: There are references to http://par.perl.org/ on CPAN, but that
website is more PARrot than PARish.
Loading...