Announcing EOL for Jessie images

[Steve McIntyre]

Hey folks,

One of our (many!) discussion topics at our sprint last week was
announcing the end of support for our published Jessie (Debian 8.x)
images. We strongly recommend that all our existing users should move
forward to Stretch (Debian 9.x) to ensure that they have continuing
security support. Jessie is no longer officially supported by Debian,
so our advertised cloud images should also reflect that.

I'm just about to remove our existing Jessie OpenStack images and move
them to the archive section on cloud.debian.org. If you have any
objections to this, please shout ASAP.

[Noah Meyerhans]

Post by Steve McIntyre
One of our (many!) discussion topics at our sprint last week was
announcing the end of support for our published Jessie (Debian 8.x)
images. We strongly recommend that all our existing users should move
forward to Stretch (Debian 9.x) to ensure that they have continuing
security support. Jessie is no longer officially supported by Debian,
so our advertised cloud images should also reflect that.

I've submitted requests to the AWS Marketplace to remove our Jessie
listings. They haven't yet acted on these requests, but they should do
so in the coming week.

I will also update the Jessie (and wheezy!?) listings on the wiki to
indicate that they're unsupported. I will remove references to specific
AMI IDs, but will include a note that previous versions of the wiki
pages contain such entries if somebody really really needs them and
understands what they're getting by launching one.

noah

[Steve McIntyre]

Post by Noah Meyerhans

Post by Steve McIntyre
One of our (many!) discussion topics at our sprint last week was
announcing the end of support for our published Jessie (Debian 8.x)
images. We strongly recommend that all our existing users should move
forward to Stretch (Debian 9.x) to ensure that they have continuing
security support. Jessie is no longer officially supported by Debian,
so our advertised cloud images should also reflect that.

I've submitted requests to the AWS Marketplace to remove our Jessie
listings. They haven't yet acted on these requests, but they should do
so in the coming week.
I will also update the Jessie (and wheezy!?) listings on the wiki to
indicate that they're unsupported. I will remove references to specific
AMI IDs, but will include a note that previous versions of the wiki
pages contain such entries if somebody really really needs them and
understands what they're getting by launching one.

Cool. I've just moved the last jessie OpenStack image into our archive
area and added a notice about the EOL in the page at

https://cloud.debian.org/images/cloud/OpenStack/

Now to disable the cron job that complains about missing updates in
those images, or I'll get all the spam... :-)

[Noah Meyerhans]

Post by Noah Meyerhans
I've submitted requests to the AWS Marketplace to remove our Jessie
listings. They haven't yet acted on these requests, but they should do
so in the coming week.

It sounds like the removal of the jessie listings has taken effect, and
this is apparently causing some pain for users. The AWS Marketplace team
has reached out to be to relay that they've been contacted by multiple
customers who are still relying on the jessie and are confused by their
disappearance. It seems that a good number of them are aware of LTS and
are expecting to make use of it.

I wonder if it might be worth it to continue to list the jessie images?

I also wonder if it might be worth it to update them with the 4.9 kernel
from LTS security? It's necessary for full KPTI, and thus the most
complete mitigations for the spectre/meltdown bugs, etc. As I understand
the concerns raised at the cloud sprint, most of them were around the
kernel. If the LTS team is keeping 4.9 fresh in jessie, these concerns
may be addressed.

As it is, a freshly booted instance of the latest published jessie AMI
has >100 outstanding package updates, so some kind of update is
definitely warranted if we're going to keep publishing them. I don't
mind doing this work, but these AMIs were created by jeb using
bootstrap-vz, and I don't know how that works or where the configuration
for them lives.

What do people think? Does anybody have particularly strong objections
to putting the AWS Marketplace listing for jessie back up? I think we
may have been hasty with the EOL of the jessie images, at least on AWS.

noah

[Noah Meyerhans]

I object because, at the 2018 Debian Cloud Sprint, we collectively
decided that we were not offering Debian LTS Cloud Images. Are we
changing our decision? I'd like to see collective decision making, not
one-offs for each platform.

That's precisely why I asked. However, I don't want to focus too much on
process or collective decision making. Every provider is different. The
existing images available for a given provider, and their adoption, is
different. The level of support given by the provider themselves is
different. The level of effort involved in continuing to support jessie
images varies by provider. Etc, etc.

I propose to revisit the decision because I think it was made in haste,
and with incomplete information. Zach expressed concern that there were
security issues not being fixed in jessie by the LTS team. My assumption
was that these issues were related specifically to the Intel speculation
class of bugs, complete mitigations for which rely on KPTI, which is not
present in the 3.16 kernels from jessie. However, it seems that LTS is
currently shipping a 4.9 kernel, in addition to the original 3.16
kernel, which was uploaded by the kernel team. (Not that you can
discover any of this on packages.d.o or similar resources, which IMO is
a real problem in terms of the legitimacy of LTS.)

If there are other specific issues that Zach (or anybody else) can point
to that haven't received attention from the LTS team, we should consider
those. Perhaps he was referring to something besides the kernel.

Given the availability of linux 4.9 in LTS, I am less attached to the
decision we made at the sprint. Given the impact that the decision has
had some some of the users of the jessie AMIs on AWS, I am interested in
revisiting it.

Note that I'm not necessarily proposing that we provide regularly
updates LTS images for the full duration of the LTS lifecycle. I'm not
actually proposing any timeline at this point, though I'll come up with
one if people want. I'm simply suggesting that we consider that we have
users who do want LTS, and we should support them to the extent that
we're able.

[Luca Filipozzi]

Post by Noah Meyerhans

I object because, at the 2018 Debian Cloud Sprint, we collectively
decided that we were not offering Debian LTS Cloud Images. Are we
changing our decision? I'd like to see collective decision making, not
one-offs for each platform.

That's precisely why I asked. However, I don't want to focus too much on
process or collective decision making. Every provider is different. The
existing images available for a given provider, and their adoption, is
different. The level of support given by the provider themselves is
different. The level of effort involved in continuing to support jessie
images varies by provider. Etc, etc.

Is this true: the goal of the Debian Cloud Team is to make roughly equievalent
Debian Cloud Images for the various cloud providers so that
Debian users have a consistent (to the degree possible) experience with
Debian in each cloud?

If true, is it also true for Debian LTS Cloud Images? I can accept that
it might not be, especially if the Debian Cloud Team _isn't_ taking
accountability for these LTS images (regardless of who is responsible
for the work: Credative for Azure, etc.).

Post by Noah Meyerhans
I propose to revisit the decision because I think it was made in haste,
and with incomplete information.

That's fair.

Post by Noah Meyerhans
Note that I'm not necessarily proposing that we provide regularly
updates LTS images for the full duration of the LTS lifecycle.

This, combined with the per-provider approach, suggests that the Debian
Cloud Team isn't accountable for the LTS images? Which would then lead
to a question about how to publish the LTS images.

Luca

[Noah Meyerhans]

Post by Luca Filipozzi
Is this true: the goal of the Debian Cloud Team is to make roughly equievalent
Debian Cloud Images for the various cloud providers so that
Debian users have a consistent (to the degree possible) experience with
Debian in each cloud?

That is a goal.

Post by Luca Filipozzi
If true, is it also true for Debian LTS Cloud Images? I can accept that
it might not be, especially if the Debian Cloud Team _isn't_ taking
accountability for these LTS images (regardless of who is responsible
for the work: Credative for Azure, etc.).

As a user, I wouldn't expect there to be any visible differences, aside
from package updates to address issues, between an oldstable image
generated during the security team's support window and an LTS image
generated after the handoff. So I don't know that I see a reason why
this *shouldn't* be a goal for the cloud team.

To a very large degree, all I would expect as a user of the LTS images
is that I have fewer pending updates on first boot than I would if I
launched the latest 8.x images generated during its oldstable window.

Clearly the project as a whole is still trying to figure out how the LTS
effort relates to the rest of the project. With a relatively small
number of people being paid to work on it, and potential a lack of
interest among the developer community in general in supporting such old
software, I can see how it might not be sustainable. However, there is
clear user demand for it, or we wouldn't be having this conversation. So
I think we're better served by figuring out how to make LTS work (in
cloud environments and more generally) than by trying to figure out how
we can say no.

Post by Luca Filipozzi

Post by Noah Meyerhans
Note that I'm not necessarily proposing that we provide regularly
updates LTS images for the full duration of the LTS lifecycle.

This, combined with the per-provider approach, suggests that the Debian
Cloud Team isn't accountable for the LTS images? Which would then lead
to a question about how to publish the LTS images.

jessie is somewhat a special case, since it completely predates the use
of FAI for our image construction, and largely predates the existence of
the cloud team in its current form. As a result, it would require
additional work on the cloud team's part in order to support it at the
same level as stretch and future releases. It's unlikely that anybody is
going to do the work to fully integrate it with the rest of the
gitlab-driven CI pipeline, so image generation may be somewhat more
manual. So today, there is additional ongoing effort required. Nobody is
required to put in that effort, but I am willing to do so in order to
support our users, and others may be as well.

For future LTS releases, ongoing support after the LTS handoff is
probably relatively easy, since we'll already have all the automation
built out and the ongoing effort will be small.

Is there any reason to expect that the LTS team will not support the
cloud team or cloud users, should we (or our members individually)
decide to continue to publish images on one or more cloud platforms?

noah

[Raphael Hertzog]

Post by Noah Meyerhans
clear user demand for it, or we wouldn't be having this conversation. So
I think we're better served by figuring out how to make LTS work (in
cloud environments and more generally) than by trying to figure out how
we can say no.

+1

Post by Noah Meyerhans
Is there any reason to expect that the LTS team will not support the
cloud team or cloud users, should we (or our members individually)
decide to continue to publish images on one or more cloud platforms?

FWIW, as I replied to Steve on debian-devel, we can certainly look
into ways for the LTS team to help the cloud team to continue to support
jessie images.

It can be in multiple ways:
- a member of the cloud team joins the LTS team as a paid contributor
for this task of managing jessie images
- a currently paid LTS contributor joins the cloud team for this task
- the LTS team builds the missing automation to help you keep jessie
images updated
- etc.

Cheers,

[Thomas Goirand]

Post by Noah Meyerhans

I object because, at the 2018 Debian Cloud Sprint, we collectively
decided that we were not offering Debian LTS Cloud Images. Are we
changing our decision? I'd like to see collective decision making, not
one-offs for each platform.

That's precisely why I asked. However, I don't want to focus too much on
process or collective decision making. Every provider is different.

We're supposed to be deciding collectively. I don't agree that every
provider is different for this specific case.

Post by Noah Meyerhans
The existing images available for a given provider, and their adoption, is
different.

No. On every platform, seeing Jessie going away is painful. Please don't
consider your own case as a special one. This isn't helping anyone, and
this isn't the case. I'd very much prefer if we were having consistency.

Cheers,

Thomas Goirand (zigo)

[Noah Meyerhans]

Post by Thomas Goirand

Post by Noah Meyerhans

I object because, at the 2018 Debian Cloud Sprint, we collectively
decided that we were not offering Debian LTS Cloud Images. Are we
changing our decision? I'd like to see collective decision making, not
one-offs for each platform.

That's precisely why I asked. However, I don't want to focus too much on
process or collective decision making. Every provider is different.

We're supposed to be deciding collectively. I don't agree that every
provider is different for this specific case.

I disagree.

Post by Thomas Goirand

Post by Noah Meyerhans
The existing images available for a given provider, and their adoption, is
different.

No. On every platform, seeing Jessie going away is painful. Please don't
consider your own case as a special one. This isn't helping anyone, and
this isn't the case. I'd very much prefer if we were having consistency.

I disagree.

The level of adoption matters, and is potentially quite
different. A newer cloud platform, or one that was not supported by
Debian, might not have any users of jessie at all. So the matter is
irrelevant to them.

The specifics of this issue relate to the discoverability and
presentation of the jessie AMIs in the AWS Marketplace. Different cloud
providers may have different systems with different behavior. For
example, there may be other ways to indicate that a release is
deprecated while still letting it show up in search results. I do not
want to have to cater to the lowest common featureset. I want to use the
mechanisms provided by the cloud provider in the native way.

noah

[David Osborne]

We are among the population of users who are still using the Jessie AMIs
and, until reading this, had planned to until 2020 due to LTS.

We often have to rebuild instances on top of your Jessie AMIs since there
are various hurdles we still have to clear before Stretch will be an option
for our whole stack.

Completely accept that limited resources mean not everything can be
supported for the same length of time, but would really like a way to keep
access to the Jessie AMIs on AWS Marketplace.

(I'd be happy just to copy it to our account but that doesn't seem possible
due to permissions)
--
David

Post by Noah Meyerhans

Post by Noah Meyerhans
I've submitted requests to the AWS Marketplace to remove our Jessie
listings. They haven't yet acted on these requests, but they should do
so in the coming week.

It sounds like the removal of the jessie listings has taken effect, and
this is apparently causing some pain for users. The AWS Marketplace team
has reached out to be to relay that they've been contacted by multiple
customers who are still relying on the jessie and are confused by their
disappearance. It seems that a good number of them are aware of LTS and
are expecting to make use of it.
I wonder if it might be worth it to continue to list the jessie images?

[Martin Zobel-Helas]

Hi David,

We are among the population of users who are still using the Jessie AMIs and, until reading this, had planned to until 2020 due to LTS.
We often have to rebuild instances on top of your Jessie AMIs since there are various hurdles we still have to clear before Stretch will be an option for our whole stack.

From my understanding Noah only removed the links from the market place, but did not remove the images from the storage. This means by knowing the image AMI IDs you should still be able to rebuild your images on top of our Debian images.

Best regards,
Martin

[David Osborne]

Thank you... so the amis themselves should remain indefinitely?
--
David

Post by Martin Zobel-Helas
From my understanding Noah only removed the links from the market place,
but did not remove the images from the storage. This means by knowing the
image AMI IDs you should still be able to rebuild your images on top of our
Debian images.
Best regards,
Martin

[Martin Zobel-Helas]

Hi David,

My understanding is that removing the images from the vendors market places is to make the images less easy discoverable and to discourage users from spinning new instances from an old image type. I don’t know the exact details for AWS but my guess is those AMI IDs will NOT remain indefinitely but at least longer.

Also be aware that we will release Debian Buster hopefully in the middle of next year. Maybe it is time to switch away from Debian Jessie at one point... Rather sooner than later...

Best regards,
Martin

Post by David Osborne
Thank you... so the amis themselves should remain indefinitely?
--
David

Post by Martin Zobel-Helas
From my understanding Noah only removed the links from the market place, but did not remove the images from the storage. This means by knowing the image AMI IDs you should still be able to rebuild your images on top of our Debian images.
Best regards,
Martin

[Martin Zobel-Helas]

David,

Also from a user’s perspective i would like to hear your feedback on the mail I wrote earlier today to the cloud list... Be aware that none of the timeline in this mail is written into stone yet. It is just a proposal.

Best regards,
Martin

Post by Martin Zobel-Helas
Hi David,
My understanding is that removing the images from the vendors market places is to make the images less easy discoverable and to discourage users from spinning new instances from an old image type. I don’t know the exact details for AWS but my guess is those AMI IDs will NOT remain indefinitely but at least longer.
Also be aware that we will release Debian Buster hopefully in the middle of next year. Maybe it is time to switch away from Debian Jessie at one point... Rather sooner than later...
Best regards,
Martin

Post by David Osborne
Thank you... so the amis themselves should remain indefinitely?
--
David

Post by Martin Zobel-Helas
From my understanding Noah only removed the links from the market place, but did not remove the images from the storage. This means by knowing the image AMI IDs you should still be able to rebuild your images on top of our Debian images.
Best regards,
Martin

[Paul Dejean]

Post by Martin Zobel-Helas
David,
Also from a user’s perspective i would like to hear your feedback on the
mail I wrote earlier today to the cloud list... Be aware that none of the
timeline in this mail is written into stone yet. It is just a proposal.
Best regards,
Martin
Hi David,
My understanding is that removing the images from the vendors market
places is to make the images less easy discoverable and to discourage users
from spinning new instances from an old image type. I don’t know the exact
details for AWS but my guess is those AMI IDs will NOT remain indefinitely
but at least longer.
Also be aware that we will release Debian Buster hopefully in the middle
of next year. Maybe it is time to switch away from Debian Jessie at one
point... Rather sooner than later...
Best regards,
Martin
Thank you... so the amis themselves should remain indefinitely?
--
David

Post by Martin Zobel-Helas
From my understanding Noah only removed the links from the market place,
but did not remove the images from the storage. This means by knowing the
image AMI IDs you should still be able to rebuild your images on top of our
Debian images.
Best regards,
Martin

Are people saying that LTS is doing a poor job of security updates?

Because I was just noticing that CVE-2017-14062 for libidn11 (not to be
confused with the more popular linidn2) fixed in 1.29-1+deb8u3 on jessie
lts but not in 1.33-1 (the current version) on stretch.

So I frankly have no idea where these "concerns were raised by several of
the cloud platforms people that LTS security doesn't seem to be working
very well" are coming from.

[David Osborne]

Thanks Martin,

We of course are in the process of migrating on from Jessie but due to LTS
the timescales were deliberately left quite loose.

Re your timeline email - I think it's a good idea having such a timeline
prominently reiterated. It certainly hadn't occurred to me that support for
debian-cloud would vary from LTS but I'm learning that one does not
necessarily follow the other.

As for LTS security support, we've found it to be excellent. The only thing
I would say (which has been mentioned already) is it wasn't a particularly
seamless transition as far as the mailing list goes.

Thanks
--
David

Post by Martin Zobel-Helas
Hi David,
My understanding is that removing the images from the vendors market
places is to make the images less easy discoverable and to discourage users
from spinning new instances from an old image type. I don’t know the exact
details for AWS but my guess is those AMI IDs will NOT remain indefinitely
but at least longer.
Also be aware that we will release Debian Buster hopefully in the middle
of next year. Maybe it is time to switch away from Debian Jessie at one
point... Rather sooner than later...
Best regards,
Martin
Thank you... so the amis themselves should remain indefinitely?
--
David

Post by Martin Zobel-Helas
From my understanding Noah only removed the links from the market place,
but did not remove the images from the storage. This means by knowing the
image AMI IDs you should still be able to rebuild your images on top of our
Debian images.
Best regards,
Martin

[Paul Dejean]

Post by David Osborne
Thanks Martin,
We of course are in the process of migrating on from Jessie but due to LTS
the timescales were deliberately left quite loose.
Re your timeline email - I think it's a good idea having such a timeline
prominently reiterated. It certainly hadn't occurred to me that support for
debian-cloud would vary from LTS but I'm learning that one does not
necessarily follow the other.
As for LTS security support, we've found it to be excellent. The only
thing I would say (which has been mentioned already) is it wasn't a
particularly seamless transition as far as the mailing list goes.
Thanks
--
David

Post by Martin Zobel-Helas
Hi David,
My understanding is that removing the images from the vendors market
places is to make the images less easy discoverable and to discourage users
from spinning new instances from an old image type. I don’t know the exact
details for AWS but my guess is those AMI IDs will NOT remain indefinitely
but at least longer.
Also be aware that we will release Debian Buster hopefully in the middle
of next year. Maybe it is time to switch away from Debian Jessie at one
point... Rather sooner than later...
Best regards,
Martin
Thank you... so the amis themselves should remain indefinitely?
--
David

Post by Martin Zobel-Helas

Post by Martin Zobel-Helas
From my understanding Noah only removed the links from the market

place, but did not remove the images from the storage. This means by
knowing the image AMI IDs you should still be able to rebuild your images
on top of our Debian images.
Best regards,
Martin

This whole thing where there's two teams with similar or overlapping

technically responsibilities (building cloud images), but who don't talk to
each other much, don't integrate technically, and in fact might make snarky
gossip asides towards each other (really people???). In the devops world
that's called siloing.

And it's something that needs to be fixed at a pretty high level of the
organization.

I'm just going to say it.

1. We should not have different mechanisms for building LTS and non LTS
images. That's so obviously technically redundant. (ie gcloud lts images
use packer, but gcloud current images use diasy, yes that's a contrived and
oversimplified example)

2. Yes Debian should maintain cloud images for releases that are in the LTS
phase.

3. LTS and cloud obviously should work on this together in harmony, rather
than tossing the football over the wall of their silo constantly.

4. I have no idea how to get this to happen, especially for a volunteer
project. It's above my paygrade!

[Noah Meyerhans]

Post by David Osborne
Thank you... so the amis themselves should remain indefinitely?

I'd expect "release" AMIs to remain indefinitely, yes. There's strong
precident for this within AWS in general; it's part of the pledge to not
break users. For example, Amazon's own Fedora Core 6 AMIs from the very
beginning of EC2 are still available and usable in all the places, with
the same AMI IDs, as ever.

What I took down the other day (and have since reverted while we
continue this discussion), was the listings within the marketplace. If
you were already using the marketplace AMIs, they'd continue working,
but you wouldn't be able to discover them using the search interface.

It's worth noting that it's never been required that you use the
marketplace. The latest AMI IDs are always shared publicly from the
Debian account. You can find the details for those on the Debian wiki:

https://wiki.debian.org/Cloud/AmazonEC2Image/

[Raphael Hertzog]

Hi Steve,

Post by Steve McIntyre
One of our (many!) discussion topics at our sprint last week was
announcing the end of support for our published Jessie (Debian 8.x)
images. We strongly recommend that all our existing users should move
forward to Stretch (Debian 9.x) to ensure that they have continuing
security support. Jessie is no longer officially supported by Debian,
so our advertised cloud images should also reflect that.

Are you unaware of Debian LTS or are there any reasons why the security
support provided by that official Debian team is not suitable in the
context of cloud images?

https://wiki.debian.org/LTS

Cheers,

[Bastian Blank]

Hi Raphael

Post by Raphael Hertzog

Post by Steve McIntyre
One of our (many!) discussion topics at our sprint last week was
announcing the end of support for our published Jessie (Debian 8.x)
images. We strongly recommend that all our existing users should move
forward to Stretch (Debian 9.x) to ensure that they have continuing
security support. Jessie is no longer officially supported by Debian,
so our advertised cloud images should also reflect that.

Are you unaware of Debian LTS or are there any reasons why the security
support provided by that official Debian team is not suitable in the
context of cloud images?

Please search for LTS in the notes of our meeting last week:

https://gobby.debian.org/export/Sprints/CloudSprint2018/Notes

Regards,
Bastian

[Raphael Hertzog]

Post by Bastian Blank

Post by Raphael Hertzog
Are you unaware of Debian LTS or are there any reasons why the security
support provided by that official Debian team is not suitable in the
context of cloud images?

https://gobby.debian.org/export/Sprints/CloudSprint2018/Notes

Thanks. Well, I don't want the cloud team to reverse their decision
but among the notes, the only valid reason I could find is
« Requires effort to keep images supported » or the fact that users
could mistakenly use old images as a basis.

I saw some (what I see as) irrelevant statement like
« Also. Debian volunteer does not provide LTS
Paid people provide support »

Or entirely wrong statement like:
« Backports (unlike LTS) is official part of Debian, hosted
on our infrastructure, maintained by DDs or DMs, etc. »

LTS is official part of Debian, hosted on debian.org, maintained
by DD. The fact that the time spent by those DD is (publicly known as
being) sponsored is irrelevant, much like the fact that the Azure support in
Debian cloud team is made by Credativ which is paid by Microsoft (IIRC).

Cheers,

[Steve McIntyre]

Post by Bastian Blank

Post by Raphael Hertzog

Post by Steve McIntyre
One of our (many!) discussion topics at our sprint last week was
announcing the end of support for our published Jessie (Debian 8.x)
images. We strongly recommend that all our existing users should move
forward to Stretch (Debian 9.x) to ensure that they have continuing
security support. Jessie is no longer officially supported by Debian,
so our advertised cloud images should also reflect that.

Are you unaware of Debian LTS or are there any reasons why the security
support provided by that official Debian team is not suitable in the
context of cloud images?

https://gobby.debian.org/export/Sprints/CloudSprint2018/Notes

The main thing: concerns were raised by several of the cloud platforms
people that LTS security doesn't seem to be working very well. They're
not seeing fixes happening for known issues, and so at the moment they
don't have trust in the process.

We therefore agreed as a team to *not* provide LTS support for cloud
images. Other people are welcome to pick up our scripts and config if
desired.

[Raphael Hertzog]

Post by Steve McIntyre
The main thing: concerns were raised by several of the cloud platforms
people that LTS security doesn't seem to be working very well. They're
not seeing fixes happening for known issues, and so at the moment they
don't have trust in the process.

Really? This is the first time I hear such feedback. Can you put me in
touch with the person(s) who made those claims so that I can try to have
more concrete information about the alleged problems?

Thank you.

[Steve McIntyre]

Post by Raphael Hertzog

Post by Steve McIntyre
The main thing: concerns were raised by several of the cloud platforms
people that LTS security doesn't seem to be working very well. They're
not seeing fixes happening for known issues, and so at the moment they
don't have trust in the process.

Really? This is the first time I hear such feedback. Can you put me in
touch with the person(s) who made those claims so that I can try to have
more concrete information about the alleged problems?

They're on the list here, hopefully ready to reply... :-)

[Marcin Kulisz]

Post by Raphael Hertzog

Post by Steve McIntyre
The main thing: concerns were raised by several of the cloud platforms
people that LTS security doesn't seem to be working very well. They're
not seeing fixes happening for known issues, and so at the moment they
don't have trust in the process.

Really? This is the first time I hear such feedback. Can you put me in
touch with the person(s) who made those claims so that I can try to have
more concrete information about the alleged problems?

It's not only about problems, there is whole paradigm shift from hand crafted
long lived servers to short lived volatile instances recreated at the whim with
no human involvement.
This drives use of latest already patched software and this includes OS, which
is treated as a cattle. In such approach LTS solutions are not necessary and
are only creating technical debt (for example migration wise).

Off course there are use cases where servers running on 'cloud' are still
treated as pets, nurtured by sysadmins etc. this is where LTS comes to play,
but those systems are long running and are neither spun up often nor in big
batches, if at all, thus doesn't really require LTS support on base images.

Having said all above it is a good practise to copy and stash your build
elements into your own environment to not depend on external resourced for CI/CD
process (even if highly reliable as Debian is). This makes base cloud images
based on old and oldold stable just additional maintenance point for Debian,
which in real life is hardly required.

Thus I'd opt for what have been done to Wheezy and Jessie. Images should be
still available but we should clearly state that those are not supported as
base OS media and that users should use latest stable instead. People who are
still going to use those IMO should be aware of EoL and informed about LTS but
that's it.

Conclusion is that IMO we shouldn't create any images for releases older than
oldstable until it's EoL and then just drop them accordingly to release cycle
of main Debian line.

[Thomas Goirand]

Post by Raphael Hertzog

Post by Steve McIntyre
The main thing: concerns were raised by several of the cloud platforms
people that LTS security doesn't seem to be working very well. They're
not seeing fixes happening for known issues, and so at the moment they
don't have trust in the process.

Really? This is the first time I hear such feedback. Can you put me in
touch with the person(s) who made those claims so that I can try to have
more concrete information about the alleged problems?
Thank you.

Raphael,

I'd very much appreciate if the LTS team decided to generate Jessie LTS
images for OpenStack clouds. But right now, I don't have the time to do
the work myself, and it looks like Steve also is busy. Could we find
some other places where to publish the image then, so that Steve doesn't
have to be involved?

Building the OpenStack Debian cloud image is super simple. It needs
running the script provided by the openstack-debian-images package.

Cheers,

Thomas Goirand (zigo)

[Martin Zobel-Helas]

Hi,

Post by Thomas Goirand
Building the OpenStack Debian cloud image is super simple. It needs
running the script provided by the openstack-debian-images package.

And testing of those images.

Best regards,
Martin

[Noah Meyerhans]

Post by Raphael Hertzog

Post by Steve McIntyre
The main thing: concerns were raised by several of the cloud platforms
people that LTS security doesn't seem to be working very well. They're
not seeing fixes happening for known issues, and so at the moment they
don't have trust in the process.

Really? This is the first time I hear such feedback. Can you put me in
touch with the person(s) who made those claims so that I can try to have
more concrete information about the alleged problems?

I'm sure a lot of it is a matter of perception, but the level of
integration of LTS with the stable lifecycle does not seem as deep as
someone familiar with Debian stable might expect it to be. For example,
security announcements being published to a list other than
debian-security-announce makes it feel very unofficial and does not
invoke the same level of confidence in the commitment (it is somewhat
remeniscent of the secure-testing effort).

Lack of integration with packages.debian.org and incomplete coverage of
the archive also present problems. For exaple, despite the existence of
DLA 1531, I cannot find evidence of a 4.9 kernel for jessie on
packages.debian.org except in jessie-backports, and backports is well
documented as not having official security support. (Again, I realize
that this may be a matter of visibility and perception.)

For my part, as maintainer of the images on AWS, I don't want to prevent
people currently using the jessie images from continuing to do so. I
simply don't want new (to AWS or to Debian) users from starting out with
jessie. As such, I've made the jessie listings slightly less
discoverable using AWS interfaces, and have noted their deprecation on
the relevant Debian wikis. Somebody who is familiar with LTS and
interested in using it is certainly welcome to do so, though.

noah

[Larry Fletcher]

Post by Noah Meyerhans
Lack of integration with packages.debian.org and incomplete coverage of
the archive also present problems. For exaple, despite the existence of
DLA 1531, I cannot find evidence of a 4.9 kernel for jessie on
packages.debian.org except in jessie-backports, and backports is well
documented as not having official security support. (Again, I realize
that this may be a matter of visibility and perception.)
For my part, as maintainer of the images on AWS, I don't want to prevent
people currently using the jessie images from continuing to do so. I
simply don't want new (to AWS or to Debian) users from starting out with
jessie. As such, I've made the jessie listings slightly less
discoverable using AWS interfaces, and have noted their deprecation on
the relevant Debian wikis. Somebody who is familiar with LTS and
interested in using it is certainly welcome to do so, though.
noah

I've been using Jessie on AWS for three years and usually check for
updates once a month. Here is the last LTS update:

Start-Date: 2018-10-16 15:21:01
Commandline: apt-get dist-upgrade
Install: linux-image-3.16.0-7-amd64:amd64 (3.16.59-1, automatic)
Upgrade: libpython2.7-stdlib:amd64 (2.7.9-2+deb8u1, 2.7.9-2+deb8u2),
libapache2-mod-php5:amd64 (5.6.
37+dfsg-0+deb8u1, 5.6.38+dfsg-0+deb8u1), python3.4:amd64 (3.4.2-1,
3.4.2-1+deb8u1), php5-common:amd6
4 (5.6.37+dfsg-0+deb8u1, 5.6.38+dfsg-0+deb8u1), python3.4-minimal:amd64
(3.4.2-1, 3.4.2-1+deb8u1), l
ibpython3.4-stdlib:amd64 (3.4.2-1, 3.4.2-1+deb8u1), python2.7:amd64
(2.7.9-2+deb8u1, 2.7.9-2+deb8u2)
, php5-readline:amd64 (5.6.37+dfsg-0+deb8u1, 5.6.38+dfsg-0+deb8u1),
php5:amd64 (5.6.37+dfsg-0+deb8u1
, 5.6.38+dfsg-0+deb8u1), libpython3.4-minimal:amd64 (3.4.2-1,
3.4.2-1+deb8u1), libxml2:amd64 (2.9.1+
dfsg1-5+deb8u6, 2.9.1+dfsg1-5+deb8u7), php5-cli:amd64
(5.6.37+dfsg-0+deb8u1, 5.6.38+dfsg-0+deb8u1),
linux-image-amd64:amd64 (3.16+63+deb8u2, 3.16+63+deb8u3),
python2.7-minimal:amd64 (2.7.9-2+deb8u1, 2
.7.9-2+deb8u2), libpython2.7-minimal:amd64 (2.7.9-2+deb8u1, 2.7.9-2+deb8u2)
End-Date: 2018-10-16 15:21:28

So far the security updates for the Jessie kernel have stayed in the
3.16.* range, and I doubt if that will change.

A few weeks ago I ran a test "dist-upgrade" to Stretch and was surprised
to see that it installed Exim. I don't use an MTA and Jessie didn't
have one when I installed it, so I guess this is something new. I
prefer a minimal image like Jessie. (If I needed an MTA I would use
Exim, but other people might want to use something else.)

So far I haven't seen any problems with Jessie and it should be okay
until June 30, 2020.

Thanks,

Larry

[Noah Meyerhans]

So far the security updates for the Jessie kernel have stayed in the 3.16.*
range, and I doubt if that will change.

Try apt-cache policy linux-image-4.9-amd64

A few weeks ago I ran a test "dist-upgrade" to Stretch and was surprised to
see that it installed Exim. I don't use an MTA and Jessie didn't have one
when I installed it, so I guess this is something new. I prefer a minimal
image like Jessie. (If I needed an MTA I would use Exim, but other people
might want to use something else.)

Exim is not installed on the stretch AMIs. If it was installed during an
upgrade, that's most likely because of a Recommends declared in another
package being installed. You can remove it.

noah

[Larry Fletcher]

Post by Noah Meyerhans

So far the security updates for the Jessie kernel have stayed in the 3.16.*
range, and I doubt if that will change.

Try apt-cache policy linux-image-4.9-amd64

I'm fine with knowing that the 3.16 kernel is getting security updates
and will probably keep using it, but I checked the server and found:

linux-image-4.9-amd64/oldstable 4.9+80+deb9u6~deb8u1 amd64

This kernel is newer than what I have on Stretch at home, but I don't
update very often. So it looks like the "Debian LTS team" is doing a
good job providing Jessie with security updates.

Thanks again,

Larry

[Raphael Hertzog]

Post by Noah Meyerhans
I'm sure a lot of it is a matter of perception, but the level of
integration of LTS with the stable lifecycle does not seem as deep as
someone familiar with Debian stable might expect it to be. For example,
security announcements being published to a list other than
debian-security-announce makes it feel very unofficial and does not
invoke the same level of confidence in the commitment (it is somewhat
remeniscent of the secure-testing effort).

I don't think that the security team is ready to open up the gate
of their list to us. But the commitment should be judged over
our longevity. It's been 5 years already and in practice we support almost
as many packages as the main security team does (because we have enough
resources nowadays).

Post by Noah Meyerhans
Lack of integration with packages.debian.org and incomplete coverage of
the archive also present problems.

packages.debian.org does cover the security suites, so LTS packages are
covered, I'm not sure what you are referring to here. The main problem
is that the updates only live on security.debian.org, they are never
merged back in the main suite (because we don't do further point
releases).

Post by Noah Meyerhans
For exaple, despite the existence of
DLA 1531, I cannot find evidence of a 4.9 kernel for jessie on
packages.debian.org except in jessie-backports, and backports is well
documented as not having official security support. (Again, I realize
that this may be a matter of visibility and perception.)

https://packages.debian.org/source/jessie/linux-4.9

Cheers,