[rt.cpan.org #102689] Module::ScanDeps isn't effective

[Roderich Schupp via RT]

Wed Mar 11 09:36:01 2015: Request 102689 was acted upon.
Transaction: Correspondence added by RSCHUPP
Queue: PAR-Packer
Subject: Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: new
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Module::ScanDeps isn't effective.

You probably meant: efficient.

Perl::LibExtractor produce much less deps tree.
It use different algorithm - "use package" in separate process and
parse %INC after - so only really used packages are found.
Also it works much more faster.

... but it doesn't catch all dependencies either.

Not that the above approach is also used by Module::ScanDeps if use
the --compile or --execute option. But the results only ADD dependencies
to the ones found during static analysis.

It will be useful to add possibility to use Perl::LibExtractor with pp
or use similar approach to build deps tree.

Code please - if you want to contribute some, add it to a wishlist bug for PAR::Packer.

Cheers, Roderich

[Karen Etheridge via RT]

Wed Mar 11 09:41:50 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ETHER
Queue: PAR-Packer
Subject: Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


There's a few different mechanisms for finding deps... Perl::PrereqScanner is another that uses PPI to find statements like 'use' and 'require' in the code. These could possibly all be used alternately or in tandem, depending on user options.

[via RT]

Wed Mar 11 09:57:29 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: PAR-Packer
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


Module::ScanDeps also didn't finds all deps, but it adds a lot of
garbage, archive size is become 10 Mb vs 5 Mb if I use libextractor.

In both cases I need to add deps manually.

Post by Roderich Schupp via RT
<URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Module::ScanDeps isn't effective.

You probably meant: efficient.

Perl::LibExtractor produce much less deps tree.
It use different algorithm - "use package" in separate process and
parse %INC after - so only really used packages are found.
Also it works much more faster.

... but it doesn't catch all dependencies either.
Not that the above approach is also used by Module::ScanDeps if use
the --compile or --execute option. But the results only ADD dependencies
to the ones found during static analysis.

It will be useful to add possibility to use Perl::LibExtractor with pp
or use similar approach to build deps tree.

Code please - if you want to contribute some, add it to a wishlist bug for PAR::Packer.
Cheers, Roderich

[via RT]

Wed Mar 11 10:04:16 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: PAR-Packer
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


Also option to add all packages under some namespace will be very useful
(like in perlapp).

For example: -M AnyEvent::*

This will allow to not to enumerate lot of packages in command line.

Post by Roderich Schupp via RT
<URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Module::ScanDeps isn't effective.

You probably meant: efficient.

Perl::LibExtractor produce much less deps tree.
It use different algorithm - "use package" in separate process and
parse %INC after - so only really used packages are found.
Also it works much more faster.

... but it doesn't catch all dependencies either.
Not that the above approach is also used by Module::ScanDeps if use
the --compile or --execute option. But the results only ADD dependencies
to the ones found during static analysis.

It will be useful to add possibility to use Perl::LibExtractor with pp
or use similar approach to build deps tree.

Code please - if you want to contribute some, add it to a wishlist bug for PAR::Packer.
Cheers, Roderich

[Roderich Schupp via RT]

Wed Mar 11 10:11:17 2015: Request 102689 was acted upon.
Transaction: Correspondence added by RSCHUPP
Queue: PAR-Packer
Subject: Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Module::ScanDeps also didn't finds all deps, but it adds a lot of
garbage, archive size is become 10 Mb vs 5 Mb if I use libextractor.

I strives to err on the side of completeness.

Cheers, Roderich

[Roderich Schupp via RT]

Wed Mar 11 10:26:21 2015: Request 102689 was acted upon.
Transaction: Correspondence added by RSCHUPP
Queue: PAR-Packer
Subject: Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Also option to add all packages under some namespace will be very
useful
(like in perlapp).
For example: -M AnyEvent::*
This will allow to not to enumerate lot of packages in command line.

On the other hand, this will produce exactly the kind of bloat you
mentioned. But I'm open to code contributions. The mechanism
for finding all installed modules in a namespace is already there
(sub _glob_in_inc), hence you only to enhance option parsing.

On the other hand, this logic is already implemented in Module::ScanDeps,
e.g. if it sees "use PPI" it will add all modules "below" PPI.
For AnyEevent there's currently no such rule, but it's a onliner for me to add one - just file a bug against Module::ScanDeps.

Cheers, Roderich

[Roderich Schupp via RT]

Wed Mar 11 10:29:19 2015: Request 102689 was acted upon.
Transaction: Correspondence added by RSCHUPP
Queue: PAR-Packer
Subject: Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by Karen Etheridge via RT
There's a few different mechanisms for finding deps...
Perl::PrereqScanner is another that uses PPI to find statements like
'use' and 'require' in the code. These could possibly all be used
alternately or in tandem, depending on user options.

Sure, IIRC pp.pm should let you "plug in" different scanner and packer modules, though it currently knows only one of each kind.

Cheers, Roderich

[via RT]

Wed Mar 11 10:44:10 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: PAR-Packer
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


Understood.

Will try to find time to make patch.

Post by Roderich Schupp via RT
<URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Also option to add all packages under some namespace will be very
useful
(like in perlapp).
For example: -M AnyEvent::*
This will allow to not to enumerate lot of packages in command line.

On the other hand, this will produce exactly the kind of bloat you
mentioned. But I'm open to code contributions. The mechanism
for finding all installed modules in a namespace is already there
(sub _glob_in_inc), hence you only to enhance option parsing.
On the other hand, this logic is already implemented in Module::ScanDeps,
e.g. if it sees "use PPI" it will add all modules "below" PPI.
For AnyEevent there's currently no such rule, but it's a onliner for me to add one - just file a bug against Module::ScanDeps.
Cheers, Roderich

[Roderich Schupp via RT]

Fri Mar 13 11:38:55 2015: Request 102689 was acted upon.
Transaction: Correspondence added by RSCHUPP
Queue: Module-ScanDeps
Subject: Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Will try to find time to make patch.

Or just try the following patch to Module::ScanDeps
which should make pp automagically include anything in namespace AnyEvent::*
(and their dependencies) without the need for any explicit -M AnyEvent::...


--- lib/Module/ScanDeps.pm (revision 1555)
+++ lib/Module/ScanDeps.pm (revision 1556)
@@ -225,6 +225,7 @@
my %Preload;
%Preload = (
'AnyDBM_File.pm' => [qw( SDBM_File.pm )],
+ 'AnyEvent.pm' => 'sub',
'Authen/SASL.pm' => 'sub',
'B/Hooks/EndOfScope.pm' => [qw( B/Hooks/EndOfScope/PP.pm B/Hooks/EndOfScope/XS.pm )],
'Bio/AlignIO.pm' => 'sub',


Cheers, Roderich

[via RT]

Wed Mar 18 04:58:04 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: Module-ScanDeps
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


Hi.

I have a question.

I use parl to make executable.

parl -B -O1.exe par.zip

Parl also pack perl dll and other binary deps.

Is it possible to pack all this deps with upx? This can reduce size on
several Mb.

Is there is any hook, that allows to do something with this files,
before parl concatenate them with exe header?

Post by Roderich Schupp via RT
<URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Will try to find time to make patch.

Or just try the following patch to Module::ScanDeps
which should make pp automagically include anything in namespace AnyEvent::*
(and their dependencies) without the need for any explicit -M AnyEvent::...
--- lib/Module/ScanDeps.pm (revision 1555)
+++ lib/Module/ScanDeps.pm (revision 1556)
@@ -225,6 +225,7 @@
my %Preload;
%Preload = (
'AnyDBM_File.pm' => [qw( SDBM_File.pm )],
+ 'AnyEvent.pm' => 'sub',
'Authen/SASL.pm' => 'sub',
'B/Hooks/EndOfScope.pm' => [qw( B/Hooks/EndOfScope/PP.pm B/Hooks/EndOfScope/XS.pm )],
'Bio/AlignIO.pm' => 'sub',
Cheers, Roderich

[via RT]

Wed Mar 18 06:25:50 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: Module-ScanDeps
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


Thanks for reply.

Under strawberry perl there are 15 standard dlls (including mingw deps),
that can be compressed from 3,2 to 1,3 total size.

Post by Roderich Schupp via RT
<URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >
Is it possible to pack all this deps with upx? This can reduce size on

Post by via RT
several Mb.

IMHO that "several Mb" claim is exaggerated. The largest file by far in
there is perl*.dll which is ~1.5 MB in size and typically would compress
around 50%.
Is there is any hook, that allows to do something with this files,

Post by via RT
before parl concatenate them with exe header?

Nope.
Cheers, Roderich

[Roderich Schupp]

On Wed, Mar 18, 2015 at 9:58 AM, ***@gmail.com via RT <
bug-Module-***@rt.cpan.org> wrote:

Is it possible to pack all this deps with upx? This can reduce size on

Post by via RT
several Mb.

IMHO that "several Mb" claim is exaggerated. The largest file by far in
there is perl*.dll which is ~1.5 MB in size and typically would compress
around 50%.

Is there is any hook, that allows to do something with this files,

Post by via RT
before parl concatenate them with exe header?

Nope.

Cheers, Roderich

[Roderich Schupp via RT]

Wed Mar 18 06:19:26 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@googlemail.com
Queue: Module-ScanDeps
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


On Wed, Mar 18, 2015 at 9:58 AM, ***@gmail.com via RT <
bug-Module-***@rt.cpan.org> wrote:

Is it possible to pack all this deps with upx? This can reduce size on

Post by via RT
several Mb.

IMHO that "several Mb" claim is exaggerated. The largest file by far in
there is perl*.dll which is ~1.5 MB in size and typically would compress
around 50%.

Is there is any hook, that allows to do something with this files,

Post by via RT
before parl concatenate them with exe header?

Nope.

Cheers, Roderich

[Roderich Schupp via RT]

Wed Mar 18 06:46:01 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: Module-ScanDeps
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Under strawberry perl there are 15 standard dlls (including mingw deps),
that can be compressed from 3,2 to 1,3 total size.

... but not worth the added complexity.

Cheers, Roderich

[via RT]

Wed Mar 18 10:35:22 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: Module-ScanDeps
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


Solution - compress dlls and then reinstall PAR::Packer.

Using upx and manual deps scanner I got 4 Mb par executable instead of
11 for complex project.

Post by Roderich Schupp via RT
<URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Under strawberry perl there are 15 standard dlls (including mingw deps),
that can be compressed from 3,2 to 1,3 total size.

... but not worth the added complexity.
Cheers, Roderich

[via RT]

Wed Mar 18 10:42:43 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: Module-ScanDeps
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >


Also, if we use rar or 7zip or something else with good compression,
instead of Archive::Zip - size can be >2Mb instead of 4.

Post by Roderich Schupp via RT
<URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Under strawberry perl there are 15 standard dlls (including mingw deps),
that can be compressed from 3,2 to 1,3 total size.

... but not worth the added complexity.
Cheers, Roderich

[Roderich Schupp via RT]

Wed Mar 18 11:07:25 2015: Request 102689 was acted upon.
Transaction: Correspondence added by ***@gmail.com
Queue: Module-ScanDeps
Subject: Re: [rt.cpan.org #102689] Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by via RT
Also, if we use rar or 7zip or something else with good compression,
instead of Archive::Zip - size can be >2Mb instead of 4.

Please stop this thread on the bug tracker - it's a BUG TRACKER, not a
general
discussion forum - and take it to the ***@perl.org mailing list instead.

Cheers, Roderich

[Slaven_Rezic via RT]

Thu Mar 19 04:05:10 2015: Request 102689 was acted upon.
Transaction: Correspondence added by SREZIC
Queue: Module-ScanDeps
Subject: Module::ScanDeps isn't effective
Broken in: (no value)
Severity: Wishlist
Owner: Nobody
Requestors: ***@cpan.org
Status: rejected
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102689 >

Post by Roderich Schupp via RT

Post by via RT
Also, if we use rar or 7zip or something else with good compression,
instead of Archive::Zip - size can be >2Mb instead of 4.

Please stop this thread on the bug tracker - it's a BUG TRACKER,

RT = Request Tracker, not bug tracker.

Post by Roderich Schupp via RT
not a
general
Cheers, Roderich

Regards,
Slaven