Hover videos for collections

[JonathanTreffler]

... like on Disney+

Disney.Plus.Hover.mp4

I think it looks awesome and i would love to have that in jellyfin.
It would probably require quite a lot of work.
But if i am not the only one, who thinks that would be awesome, i would be willing to do the dev work to make this reality.
I don't think transcoding would be necessary.
The hard part would probably be to figure out how the admin can add these. Upload from the UI like with custom posters ?
I have no idea how much work would be needed for the server to handle the upload of these videos.
The frontend implementation should be fairly easy.
It would not be worth the work to backport this to the old web client, so this might be something that can only be realized after this client is the default, wich will take quite a while.

[Perseusdehond]

I would like something like that, although I don't know what the best implementation should be.

For myself it wouldn't be so convenient to put the work into making such a video, although it is cool (and you can probably get a video online depending on the property)

Maybe you could add such a feature, but when hovering over an collection, it makes the transition to different posters/background depending on the orientation of the box and with the images from movies in that collection automatically random (or selected out of best rated, favourites etc. e.g.)

Then you can choose somewhere in the settings to change between a few transition types when hovering over a collection.

I feel something like that would be easier and less work for people to make look good I guess.

[JonathanTreffler]

and you can probably get a video online depending on the property

The one that's in the example video can be downloaded easily from Disney+. It has no copy protection. It is just a video element with a mp4 file :)

[JonathanTreffler]

Maybe you could add such a feature, but when hovering over an collection, it makes the transition to different posters/background depending on the orientation of the box and with the images from movies in that collection automatically random (or selected out of best rated, favourites etc. e.g.)

I would not personally use that, because i don't really see any value to the user experience, but this seems like once hover video support would be implemented it would be easy to develop as a Plugin, that automatically generates such videos.

[GlassedSilver]

Couldn't have put it better myself, @JonathanTreffler!

[heyhippari]

This could likely be done with generic animated posters, no need for something bespoke.

Though it would need some changes to the server in order to get something workable.

Here's the gist of what I have in mind:

Primary images stay like they are now, to keep compatibility with every client. If uploading a GIF, the server would use the first frame for this.
A new image type, hidden from the user (Animated Primary for example) is added and contains the actual animation (GIF, WebP, APNG or whatever format).

On hover, if there is an animated version of the poster, we switch over to that, then switch back on mouseout, giving the effect shown above.

This technique is doable on pretty much all platforms, even low power ones and is pretty simple to implement (at least client side. Adding stuff to the server is always painful in unexpected ways).

[JonathanTreffler]

The server just needs to statically serve the video files, handle the storage and the upload through the ui.
It's just the same as serving an animated image.

[JonathanTreffler]

The system itself should be format-agnostic.

Absolutely

[heyhippari]

Transcoding is in my opinion not necessary. A user should probably not use any video with more than 420p, but it is up to them.
I am not too woried about H264 in mp4/mkv compatibility.

Well, you should be worried about it, because plenty of things don't support MKV. Browsers, for example. We can't assume that platforms will support every kind of videos, and we can't require people to only upload these in a specific format (Since it all has to integrate with potential image providers, down the line, and these might do things differently).

There are also limitations around automatic playback of videos on some platforms, notably Safari.

Plus, as I mentioned before, not all platforms allow you to embed a video into a card or something in a menu, while pretty much everything allows using images.

But an animated webp image ? I would be worried about compatibility there.

We already have API support for converting images to another format and resizing them. Adding animation support to that should be fairly trivial, if it's not already there. We also do feature detection pretty extensively on the current web client, and have stuff in place to do it in Vue as well. So, worst case scenario, we can disable animated posters entirely, if need be.

The server just needs to statically serve the video files, handle the storage and the upload through the ui.
It's just the same as serving an animated image.

Well, that's also a thing: we don't have anything to do this at the moment, while images are already widely supported across the board. It avoids introducing more code, more client complexity (We'll need two card image components, since one will have a video and the other an image...), more API complexity (You'll have two ways to handle videos, one for animated posters and one for actual videos...)

Disney+, Netflix and all have one advantage we don't have: they curate their content and can pre-generate compatible formats for everything they support. We don't.

We need to be able to ensure that whatever we send will run on that device, which images do pretty nicely (Since we get that info via headers and already do processing server-side with caching and all).

For videos, whatever the scenario is, we will need transcoding support, which means that currently we need to ask for a playback session, provide a device profile and so on.

So, for Jellyfin, it's pretty clear that going with animated images is the best solution possible, imo.

[JonathanTreffler]

We already have API support for converting images to another format and resizing them. Adding animation support to that should be fairly trivial, if it's not already there. We also do feature detection pretty extensively on the current web client, and have stuff in place to do it in Vue as well. So, worst case scenario, we can disable animated posters entirely, if need be.

Well your a maintainer, so if you say so 😃

[GlassedSilver]

Animated images just places the burden on the creators of these kinds of fanarts to convert them and since sourcing appropriate kinds of files will often lead to a user creating them themselves at least for quite some time until either fanart.tv or other similar places add support for this and make it easily scrapable this is definitely a factor.

Maybe this could also be factored into the same backend mechanism that would be used to generate optimized versions of the movies/shows themselves Plex-style.

Let me dig up the issue for this...

Okay not an issue, but a voted feature request: https://features.jellyfin.org/posts/284/convert-option-similar-to-plexs-optimise