[rt.cpan.org #132811] Win32: Crash a week after start

Fri Jun 12 19:44:29 2020: Request 132811 was acted upon.
Transaction: Correspondence added by SLAFFAN
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: new
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

This is related to https://rt.cpan.org/Public/Bug/Display.html?id=101800

Shawn.

Post by Neubauer, Ralf via RT
Hi,
I just found out why PAR::Packer-generated executables tend to crash
when I interact with them after I started them some days ago and they
stayed mostly idle in the task bar -- this also happened on Win7, but
https://support.microsoft.com/en-us/help/4506040/temp-folder-with-
logon-session-id-is-deleted-unexpectedly
You can imagine a program behaving strange or crashing when it wants
to load modules, resources or data files from its PAR_TEMP directory,
but everything but the already open DLLs has vanished. I don't know
exactly which windows versions are affected, but I don't want to give
programs to users that suddenly crash out of nowhere, they might lose
trust into the programs.
I don't know if there can be a better solution than documenting this
behaviour. Of course one can set PAR_GLOBAL_TEMP or PAR_GLOBAL_TMPDIR,
but you have to use wrapper scripts or an installer that find a
suitable directory on the user's machine and set the variables for the
process or globally -- which a bit defeats the 'one executable, no
installer, just double-click the file I gave you' purpose of
PAR::Packer. So the workaround I found is (without the Win32API::File-
if ('CODE' eq ref $INC[-1] && $^O eq 'MSWin32' &&
!$ENV{PAR_GLOBAL_TEMP} && !$ENV{PAR_GLOBAL_TMPDIR}) {
use File::Find;
use Win32API::File qw(:ALL);
#use threads;
#(async
{
no warnings 'File::Find';
find +{
no_chdir => 1,
wanted => sub {
# ignore result, handle stays open until program
terminates
-f and createFile $_, 'r ke', 'rw';
},
}, $ENV{PAR_TEMP};
}
#)->detach;
}
Is there a better way?
Mit freundlichen Grüßen
Ralf

Neubauer, Ralf via RT

2020-06-13 01:35:13 UTC

Fri Jun 12 21:35:12 2020: Request 132811 was acted upon.
Transaction: Correspondence added by ***@wido.bv.aok.de
Queue: PAR-Packer
Subject: RE: [rt.cpan.org #132811] Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

Hi.

Post by Shawn Laffan via RT
This is related to https://rt.cpan.org/Public/Bug/Display.html?id=101800

Yes and no. _CANARY_.txt exists and the directory will be repopulated on the next start of the program, so there is no problem except having to unpack the files again between runs of the program. My problem on the other hand is, that I start the program and the files are deleted while it is running. My crude protection scheme (locking all of the files -- 3500 files in my case, which only works with Windows file handles since Perl only gets the 2048 file handles implemented by the Windows libc) only protects the files while the program is running, so _CANARY_.txt is still needed (and there is a short time window at the start of the program, inbetween starting the unpacking and locking the files, where files could still vanish if you are unlucky enough to have cleanmgr.exe run at exactly the same time.

Using a non-temp-directory would solve both problems but carries some problems on its own, see original message. Declaring %TEMP% to be a non-temp directory sounds strange. Raising the existence interval only makes it less bad, but doesn't solve it. Also you have to have administrator privilege for both.

I also thought about keeping the files fresh by manipulating their timestamps, that also doesn't really solve it, but reflects the truth that the files have been used very recently. The would be easy but time-consuming on startup (and make _CANARY_.txt obsolete). To solve the runtime problems you would have to spawn a parallel job to keep the files fresh every couple of hours.

In essence, the reason is the same, but I don't see a simultaneous solution for both aspects that doesn't require user oder administrator input.

Mit freundlichen Grüßen

Ralf

Shawn Laffan via RT

2020-06-13 02:04:57 UTC

Fri Jun 12 22:04:55 2020: Request 132811 was acted upon.
Transaction: Correspondence added by SLAFFAN
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

Post by Shawn Laffan via RT
This is related to
https://rt.cpan.org/Public/Bug/Display.html?id=101800

Yes and no. _CANARY_.txt exists and the directory will be repopulated
on the next start of the program, so there is no problem except having
to unpack the files again between runs of the program. My problem on
the other hand is, that I start the program and the files are deleted
while it is running. My crude protection scheme (locking all of the
files -- 3500 files in my case, which only works with Windows file
handles since Perl only gets the 2048 file handles implemented by the
Windows libc) only protects the files while the program is running, so
_CANARY_.txt is still needed (and there is a short time window at the
start of the program, inbetween starting the unpacking and locking the
files, where files could still vanish if you are unlucky enough to
have cleanmgr.exe run at exactly the same time.
Using a non-temp-directory would solve both problems but carries some
problems on its own, see original message. Declaring %TEMP% to be a
non-temp directory sounds strange. Raising the existence interval only
makes it less bad, but doesn't solve it. Also you have to have
administrator privilege for both.
I also thought about keeping the files fresh by manipulating their
timestamps, that also doesn't really solve it, but reflects the truth
that the files have been used very recently. The would be easy but
time-consuming on startup (and make _CANARY_.txt obsolete). To solve
the runtime problems you would have to spawn a parallel job to keep
the files fresh every couple of hours.
In essence, the reason is the same, but I don't see a simultaneous
solution for both aspects that doesn't require user oder administrator
input.
Mit freundlichen Grüßen
Ralf

Yes, the canary file only helps in cases where files are deleted between program runs or where files only need to be read at startup.

The regular temp file cleanup will also impact all users in that the PAR archive needs to be unpacked each time it is cleaned up, causing a slow user experience for large archives at startup.

FWIW, I'd be OK if the default PAR_TEMP location was in the user's home dir or under C:\ProgramData ($ENV{PROGRAMDATA}). The latter might have issues on shared machines depending on how the unpacked files are used, though.

Shawn.

Oliver Betz

2020-06-13 05:36:14 UTC

Post by Neubauer, Ralf via RT
Is there a better way?

did you consider to use pp to crete the set of required files, but *not*
at runtime?

You can then put all the required files to a safe directory, optionally
even write protected for ordinary users to avoid unexpected modification.

I made a simple launcher replacing perl.exe if your users don't want to
need to invoke perl.exe yourapplication.pl

I remember there is even an "unpacker" from another user to extract all
the needed files from a pp created exe archive.

Oliver

Shawn Laffan via RT

2020-06-13 06:59:54 UTC

Sat Jun 13 02:59:53 2020: Request 132811 was acted upon.
Transaction: Correspondence added by SLAFFAN
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

Post by Shawn Laffan via RT
This is related to
https://rt.cpan.org/Public/Bug/Display.html?id=101800

Yes and no. _CANARY_.txt exists and the directory will be repopulated
on the next start of the program, so there is no problem except
having
to unpack the files again between runs of the program. My problem on
the other hand is, that I start the program and the files are deleted
while it is running. My crude protection scheme (locking all of the
files -- 3500 files in my case, which only works with Windows file
handles since Perl only gets the 2048 file handles implemented by the
Windows libc) only protects the files while the program is running,
so
_CANARY_.txt is still needed (and there is a short time window at the
start of the program, inbetween starting the unpacking and locking
the
files, where files could still vanish if you are unlucky enough to
have cleanmgr.exe run at exactly the same time.
Using a non-temp-directory would solve both problems but carries some
problems on its own, see original message. Declaring %TEMP% to be a
non-temp directory sounds strange. Raising the existence interval
only
makes it less bad, but doesn't solve it. Also you have to have
administrator privilege for both.
I also thought about keeping the files fresh by manipulating their
timestamps, that also doesn't really solve it, but reflects the truth
that the files have been used very recently. The would be easy but
time-consuming on startup (and make _CANARY_.txt obsolete). To solve
the runtime problems you would have to spawn a parallel job to keep
the files fresh every couple of hours.
In essence, the reason is the same, but I don't see a simultaneous
solution for both aspects that doesn't require user oder
administrator
input.
Mit freundlichen Grüßen
Ralf

Yes, the canary file only helps in cases where files are deleted
between program runs or where files only need to be read at startup.
The regular temp file cleanup will also impact all users in that the
PAR archive needs to be unpacked each time it is cleaned up, causing a
slow user experience for large archives at startup.
FWIW, I'd be OK if the default PAR_TEMP location was in the user's
home dir or under C:\ProgramData ($ENV{PROGRAMDATA}). The latter
might have issues on shared machines depending on how the unpacked
files are used, though.
Shawn.

$ENV{APPDATA} would be better than $ENV{PROGRAMDATA}

It is easy enough to add to mktmpdir.c for Windows builds, so I can work up a PR if Roderich is in favour of the idea.

Neubauer, Ralf via RT

2020-06-13 10:44:04 UTC

Sat Jun 13 06:44:03 2020: Request 132811 was acted upon.
Transaction: Correspondence added by ***@wido.bv.aok.de
Queue: PAR-Packer
Subject: RE: [rt.cpan.org #132811] Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

Hi.

Post by Shawn Laffan via RT
FWIW, I'd be OK if the default PAR_TEMP location was in the user's
home dir or under C:\ProgramData ($ENV{PROGRAMDATA}). The latter
might have issues on shared machines depending on how the unpacked
files are used, though.

$ENV{APPDATA} would be better than $ENV{PROGRAMDATA}
It is easy enough to add to mktmpdir.c for Windows builds, so I can work up a PR if Roderich is in favour of the idea.

What is your concept for getting rid of old PAR_TEMP locations once you get (or create) an updated version of the application or just stop using it? Explicit deinstallation (maybe by using a special argument or variable when calling this version)? An own version of 'delete old files' but with a longer timeout and using a better protocol for detecting the last use and removing only whole directories? Giving the user better hints than cryptic directory names with cryptic files in it, that he can decide which dirs to purge manually? This could be implemented by description files or by a version manager.

Explicit deinstallation could be paired with explicit installation -- the application only unpacks intelf into a permanent location, if you explicely tell it to do so, in a way every user understands.

Mit freundlichen Grüßen

Ralf Neubauer

Oliver Betz

2020-06-14 19:18:47 UTC

Neubauer, Ralf wrote:

[PAR_TEMP]

Post by Neubauer, Ralf via RT
What is your concept for getting rid of old PAR_TEMP locations once you get (or create) an updated version of the application or just stop using it?

My simple concept is not to hide the unpacking but tell the users the truth:

There is a bunch of files needed to run the Perl application, and if you
don't use it anymore, you need to remove it.

I decided to use the PAR packager only to collect the needed files but
not to use it's runtime magic.

You find an extraction tool at
https://github.com/ChordPro/chordpro/blob/dev/pp/pp2ppl.pl

My primitive launcher is here:
https://oliverbetz.de/pages/Artikel/Portable-Perl-Applications

If you want to provide an uninstaller, use Inno Setup, NSIS,
InstallShield or something similar for a clean solution.

Oliver

Shawn Laffan via RT

2020-06-13 22:45:04 UTC

Sat Jun 13 18:45:02 2020: Request 132811 was acted upon.
Transaction: Correspondence added by SLAFFAN
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

$ENV{APPDATA} would be better than $ENV{PROGRAMDATA}
It is easy enough to add to mktmpdir.c for Windows builds, so I can
work up a PR if Roderich is in favour of the idea.

What is your concept for getting rid of old PAR_TEMP locations once
you get (or create) an updated version of the application or just stop
using it? Explicit deinstallation (maybe by using a special argument
or variable when calling this version)? An own version of 'delete old
files' but with a longer timeout and using a better protocol for
detecting the last use and removing only whole directories? Giving the
user better hints than cryptic directory names with cryptic files in
it, that he can decide which dirs to purge manually? This could be
implemented by description files or by a version manager.
Explicit deinstallation could be paired with explicit installation --
the application only unpacks intelf into a permanent location, if you
explicely tell it to do so, in a way every user understands.
Mit freundlichen Grüßen
Ralf Neubauer

Currently there is no removal of old PAR archives that I am aware of. Removal is left to the user or automated processes.

It could be done manually using PAR_GLOBAL_CLEAN at the cost of running the exe. e.g. for some_archive.exe this could be packed into an uninstall.bat:

set PAR_GLOBAL_CLEAN=1
some_archive.exe

Maybe this could be added to PAR so it does not run the exe and just does the cleanup, e.g.:

some_archive.exe --run-par-cleanup

It's probably conceptually simpler for the user if it is a separate process, though.

It would also be possible to add that logic to your own perl script so it can exit without doing anything, but it might be too late by the time that is loaded.

Shawn Laffan via RT

2020-06-14 03:32:48 UTC

Sat Jun 13 23:32:46 2020: Request 132811 was acted upon.
Transaction: Correspondence added by SLAFFAN
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

Post by Shawn Laffan via RT
FWIW, I'd be OK if the default PAR_TEMP location was in the
user's
home dir or under C:\ProgramData ($ENV{PROGRAMDATA}). The latter
might have issues on shared machines depending on how the
unpacked
files are used, though.

$ENV{APPDATA} would be better than $ENV{PROGRAMDATA}
It is easy enough to add to mktmpdir.c for Windows builds, so I can
work up a PR if Roderich is in favour of the idea.

What is your concept for getting rid of old PAR_TEMP locations once
you get (or create) an updated version of the application or just
stop
using it? Explicit deinstallation (maybe by using a special argument
or variable when calling this version)? An own version of 'delete old
files' but with a longer timeout and using a better protocol for
detecting the last use and removing only whole directories? Giving
the
user better hints than cryptic directory names with cryptic files in
it, that he can decide which dirs to purge manually? This could be
implemented by description files or by a version manager.
Explicit deinstallation could be paired with explicit installation --
the application only unpacks intelf into a permanent location, if you
explicely tell it to do so, in a way every user understands.
Mit freundlichen Grüßen
Ralf Neubauer

Currently there is no removal of old PAR archives that I am aware of.
Removal is left to the user or automated processes.
It could be done manually using PAR_GLOBAL_CLEAN at the cost of
running the exe. e.g. for some_archive.exe this could be packed into
set PAR_GLOBAL_CLEAN=1
some_archive.exe
Maybe this could be added to PAR so it does not run the exe and just
some_archive.exe --run-par-cleanup
It's probably conceptually simpler for the user if it is a separate
process, though.
It would also be possible to add that logic to your own perl script so
it can exit without doing anything, but it might be too late by the
time that is loaded.

I've run some testing and neither approach works. When PAR_GLOBAL_TEMP is set before running the exe, the archive is unpacked into a different folder named like temp-12345 and cleaned up on exit. Setting it in a BEGIN block within the packed script appears too late to have any effect. Presumably the environment is localised or some other flag is set by PAR at startup to check on exit.

It looks like a separate uninstaller is needed, although it would need to ensure it does not remove files from under a running process, as otherwise it is another example of the original reported issue.

Shawn.

Roderich Schupp via RT

2020-06-14 15:24:20 UTC

Sun Jun 14 11:24:18 2020: Request 132811 was acted upon.
Transaction: Correspondence added by RSCHUPP
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

Post by Shawn Laffan via RT
$ENV{APPDATA} would be better than $ENV{PROGRAMDATA}
It is easy enough to add to mktmpdir.c for Windows builds, so I can
work up a PR if Roderich is in favour of the idea.

Go for it :) I suggest "$ENV{APPDATA}/Local/pp" and maybe we may omit the per-user sub directory
in this case (as %APPDATA% should already be per-user). Maybe drop the ancient fallback to C:\TEMP
(and the really weird %WinDir%\temp) as well.

Keep in mind that par_mktmpdir in myldr/mktmpdir.c has a perl-level sibling _set_par_temp in script/par.pl.

Cheers, Roderich

Roderich Schupp via RT

2020-06-14 15:49:05 UTC

Sun Jun 14 11:49:03 2020: Request 132811 was acted upon.
Transaction: Correspondence added by RSCHUPP
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: ***@wido.bv.aok.de
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >

Post by Neubauer, Ralf via RT
What is your concept for getting rid of old PAR_TEMP locations once
you get (or create) an updated version of the application or just stop
using it?

There is none.

Post by Neubauer, Ralf via RT
Explicit deinstallation (maybe by using a special argument
or variable when calling this version)?

Nope. PAR::Packer is expressly designed to work WITHOUT installation and tacking on stuff
is out of the question - there are already too many special arguments/variables.
If you want installation/deinstallation look somewhere else.

Cheers, Roderich

Neubauer, Ralf via RT

2020-06-14 20:21:46 UTC