bug#31065: Version 2??

[- -]

Hi, some time ago Jean-loup, said on [1]http://www.gzip.org/recover.txt
that
"As you can see, all this is not a trivial task, so you should attempt
it only if your data is very valuable. gzip 2.0 will have a new
blocksize option, allowing to recover easily all undamaged blocks after
the damaged portion."
I'm using gzip 1.9 from the PCLinuxOS.
Can you please let me know:
1 Is it still the plan to get this into 2.0?
2 If you are, when do you think 2.0 will be out??
Thanks loads
Martin

References

1. http://www.gzip.org/recover.txt

[Mark Adler]

Jean-loup has not worked on gzip for many years, but I will leave it to the gzip maintainers here to answer to their future intentions.

However pigz has that ability now with the --independent option, where the block size defaults to 128K, and can be changed with the --blocksize option. See http://zlib.net/pigz/

Post by - -
Hi, some time ago Jean-loup, said on [1]http://www.gzip.org/recover.txt
that
"As you can see, all this is not a trivial task, so you should attempt
it only if your data is very valuable. gzip 2.0 will have a new
blocksize option, allowing to recover easily all undamaged blocks after
the damaged portion."
I'm using gzip 1.9 from the PCLinuxOS.
1 Is it still the plan to get this into 2.0?
2 If you are, when do you think 2.0 will be out??
Thanks loads
Martin
References
1. http://www.gzip.org/recover.txt

[Jim Meyering]

tags 31065 notabug
stop

Post by Mark Adler
Jean-loup has not worked on gzip for many years, but I will leave it to the gzip maintainers here to answer to their future intentions.
However pigz has that ability now with the --independent option, where the block size defaults to 128K, and can be changed with the --blocksize option. See http://zlib.net/pigz/

Hi Mark, thanks for replying.

As for gzip vs. 2.0, I can say with confidence that we would strongly
discourage such an effort here. If you want that capability, use pigz.
While gzip is worth maintaining, it is definitely not the compression
tool of the future.

I've marked this as "notabug" and closed the issue in gzip's tracker,
but you're welcome to continue replying here.

[Mark Adler]

Jim,

So gzip has run into a version 2.0 wall. Just out of curiosity, will the next version be 1.91? 2.0? 1.A?

Mark

Post by Jim Meyering
tags 31065 notabug
stop

Post by Mark Adler
Jean-loup has not worked on gzip for many years, but I will leave it to the gzip maintainers here to answer to their future intentions.
However pigz has that ability now with the --independent option, where the block size defaults to 128K, and can be changed with the --blocksize option. See http://zlib.net/pigz/

Hi Mark, thanks for replying.
As for gzip vs. 2.0, I can say with confidence that we would strongly
discourage such an effort here. If you want that capability, use pigz.
While gzip is worth maintaining, it is definitely not the compression
tool of the future.
I've marked this as "notabug" and closed the issue in gzip's tracker,
but you're welcome to continue replying here.

[Jim Meyering]

Post by Mark Adler
So gzip has run into a version 2.0 wall. Just out of curiosity, will the next version be 1.91? 2.0? 1.A?

I might have blindly chosen 1.10, since most are used to
version-sorting for such numbers. But these version-number-lengthening
events do make one think: not too long ago, I "incremented" 2.28 to
3.0 for grep. Now that you've raised the issue, maybe 2.0 to keep it
short and simple.

[Paul Eggert]

Post by Jim Meyering
maybe 2.0 to keep it
short and simple.

I have a more-drastic idea in mind. Let's replace gzip's source code
with pigz's, make the minimal set of changes needed to make it
compatible with gzip and/or GNU in general, and call it gzip 2.0. In the
meantime, we can keep the gzip 1 series around with the traditional
implementation.

[Jim Meyering]

Post by Jim Meyering
maybe 2.0 to keep it
short and simple.

I have a more-drastic idea in mind. Let's replace gzip's source code with
pigz's, make the minimal set of changes needed to make it compatible with
gzip and/or GNU in general, and call it gzip 2.0. In the meantime, we can
keep the gzip 1 series around with the traditional implementation.

I like it!

[Mark Adler]

Jim,

I’d certainly support that, but it would take some work to make pigz more portable. It depends on the POSIX pthread functions, where I don’t know how that will play out on, for example, Windows. I have an Android report where apparently pthread is not quite the same. Also the pigz Makefile is pretty simple, and there is no configure for where there might be system dependencies that need to be remedied.

As a consequence, there would need to be a fair bit of testing to make sure it works across a wide variety of systems. The current gzip has the advantage of having been deployed over a very wide range of systems over a long time, so a lot of portability issues have been worked out.

Mark

Post by Jim Meyering

Post by Jim Meyering
maybe 2.0 to keep it
short and simple.

I have a more-drastic idea in mind. Let's replace gzip's source code with
pigz's, make the minimal set of changes needed to make it compatible with
gzip and/or GNU in general, and call it gzip 2.0. In the meantime, we can
keep the gzip 1 series around with the traditional implementation.

I like it!

[Paul Eggert]

Post by Mark Adler
As a consequence, there would need to be a fair bit of testing to make sure it works across a wide variety of systems. The current gzip has the advantage of having been deployed over a very wide range of systems over a long time, so a lot of portability issues have been worked out.

Absolutely. I expect that most of the work will be testing and
portability-enhancement. For example, we could use the Gnulib pthreads
module to insulate the gzip code proper from the vagaries of threads on
non-POSIX platforms (this won't work as well as native threads, but
that's OK, gzip will still run).

[Mark Adler]

gzip overlords,

Jean-loup has retired his old gzip.org site, and pointed the domain to my service to host it. I have created a new, brief gzip page there. Please take a look and let me know if you have any suggested corrections, improvements, or additions. Thanks.

https://gzip.org/

Mark

[Paul Eggert]

Thanks for the heads-up. It looks good. Perhaps "woolly mammoth" could link to:

http://sitn.hms.harvard.edu/flash/2017/woolly-mammoths-walk/

as a sign that even that comment may become out-of-date? :-)

[Garreau\, Alexandre]

Sorry, in my excitement, I sent my last mail without looking at what I
wanted to say/ask in my first draft:

I’m not sure of recalling or understanding fully, but isn’t pigz
somewhat linked with Zlib/Zlib code? and Zlib not being GNU, what would
that mean? would it have to be separated? would gzip get Zlib as a
mandatory dependency? How would that evolve?

As I understood anyway you Mark Adler the maintainer of Zlib are anyway
quite active on these mailing-lists. Out of curiosity, and I’m probably
bad at formulating it at a quick glance but I’m not sure of what are the
relationships and differences between zlib and the (redundant?
different? independant?) standalone compression tools it reimplements
(or the other way around?), including gzip, especially when some people
working on all these are the same: then there must be some relevant
useful difference that justify the differenciation of both?

I first learnt about pigz on a (french) (micro)blogger website [1], then
I was really curious about why a such useful and uncontroversial change
wouldn’t go upstream (as afaik stuff not going upstream for gcc or glibc
has been common for several times in their history), not only for gzip
but also for lzip, xz, bzip2, etc.

Thank you in advance for any answer!

[1] fr: <http://sebsauvage.net/links/?DrNtGw>

[Mark Adler]

pigz isn’t under GPL either. It has the same zlib license that zlib has. Interestingly the “zlib license” has become a thing onto itself and used by others, as one of the approved licenses by FSF and OSI. FSF calls the zlib license “GPL compatible”, whatever that means.

For pigz to eventually replace and be called “gzip", does it need to be under GPL?

As to the history of gzip and zlib, gzip was written first, itself derived from earlier Info-ZIP code. zlib was written a few years later by the same authors for compression and decompression, but they both made changes to the code and algorithms. So, for example, the compressed data that comes out of gzip will generally be different than for zlib, for the same input and compression level. They are completely compatible with each other, conforming to the deflate format. I am not aware of any reason that gzip couldn’t use zlib, but it just happens not to.

Post by Garreau\, Alexandre
Sorry, in my excitement, I sent my last mail without looking at what I
I’m not sure of recalling or understanding fully, but isn’t pigz
somewhat linked with Zlib/Zlib code? and Zlib not being GNU, what would
that mean? would it have to be separated? would gzip get Zlib as a
mandatory dependency? How would that evolve?
As I understood anyway you Mark Adler the maintainer of Zlib are anyway
quite active on these mailing-lists. Out of curiosity, and I’m probably
bad at formulating it at a quick glance but I’m not sure of what are the
relationships and differences between zlib and the (redundant?
different? independant?) standalone compression tools it reimplements
(or the other way around?), including gzip, especially when some people
working on all these are the same: then there must be some relevant
useful difference that justify the differenciation of both?
I first learnt about pigz on a (french) (micro)blogger website [1], then
I was really curious about why a such useful and uncontroversial change
wouldn’t go upstream (as afaik stuff not going upstream for gcc or glibc
has been common for several times in their history), not only for gzip
but also for lzip, xz, bzip2, etc.
Thank you in advance for any answer!
[1] fr: <http://sebsauvage.net/links/?DrNtGw>

[Paul Eggert]

Just as a heads-up, I've assigned to some of my students the job of
rewriting gzip to use zlib; other way of putting it is to redo pigz to
be as gzip-compatible as possible and to make it as portable as possible
to non-POSIX environments. Early days yet.

[Garreau\, Alexandre]

Post by Paul Eggert

Post by Jim Meyering
maybe 2.0 to keep it
short and simple.

I have a more-drastic idea in mind. Let's replace gzip's source code
with pigz's, make the minimal set of changes needed to make it
compatible with gzip and/or GNU in general, and call it gzip 2.0. In
the meantime, we can keep the gzip 1 series around with the
traditional implementation.

I totally support this idea too!

that’s at least… some months I was collecting a maildir of all the mail
mentioning pigz and the future of gzip here and there, waiting to be put
in the References header of a mail I was still composing to ask what was
gzip going to be now there’s pigz that seems to replace everything, and
be maintained by people active here, and I’m ignoring all this just to
say +1 to this.