Support for Grays wrapping signed FixedPointNumbers

[tlnagy]

Currently ImageShow breaks when trying to display images with signed FixedPointNumbers, e.g. Q0f31, etc

julia> using TiffImages, ImageShow

julia> img = TiffImages.load("example.tif")
┌ Info: For better quality inline display of large images or thumbnails, load the Images package.
└ @ ImageShow /home/tlnagy/.julia/packages/ImageShow/11zF7/src/showmime.jl:131
┌ Info: Precompiling ImageMagick [6218d12a-5da1-5696-b52f-db25d2ecc6d1]
└ @ Base loading.jl:1317
┌ Warning: Mapping to the storage type failed; perhaps your data had out-of-range values?
│ Try `map(clamp01nan, img)` to clamp values to a valid range.
└ @ ImageMagick /home/tlnagy/.julia/packages/ImageMagick/b8swT/src/ImageMagick.jl:180
Errors encountered while save FileIO.Stream{FileIO.DataFormat{:PNG}, IOContext{Base64.Base64EncodePipe}, Nothing}(IOContext(Base64.Base64EncodePipe(IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1), 
All errors:
===========================================
ArgumentError: FixedPointNumbers.Q0f31 is a 32-bit type representing 4294967296 values from -1.0 to 1.0; cannot represent 1
===========================================
ArgumentError: FixedPointNumbers.Q0f31 is a 32-bit type representing 4294967296 values from -1.0 to 1.0; cannot represent 1
===========================================
FileIO.MimeWriter writer error: neither save nor fileio_save is defined
  due to 
...

example.tif is from tlnagy/TiffImages.jl#56 (comment)

julia> eltype(img)
ColorTypes.Gray{FixedPointNumbers.Q0f31}

It would super nice to display these sorts of images properly, i.e. with one color for negative numbers and another for positive or even reinterpreted as normed FixedPointNumbers. The former is definitely preferable, but the current behavior isn't nice for the end user.

[tlnagy]

Here's a shorter example

julia> using ImageShow, FixedPointNumbers, Colors                                                                                                                       
                                                                                                                                                             
julia> rand(Gray{Q0f31}, 100, 100)    
All errors:
===========================================
ArgumentError: Q0f31 is a 32-bit type representing 4294967296 values from -1.0 to 1.0; cannot represent 1
===========================================
ArgumentError: Q0f31 is a 32-bit type representing 4294967296 values from -1.0 to 1.0; cannot represent 1
===========================================
FileIO.MimeWriter writer error: neither save nor fileio_save is defined
  due to MethodError(FileIO.MimeWriter.var"#save##kw"(), ((mapi = ImageShow.var"#8#10"(),), FileIO.MimeWriter.save, FileIO.Stream{FileIO.DataFormat{:PNG}, IOContext{Base64.Base64EncodePipe}, Nothing}(IOContext(Base64.Base64EncodePipe(IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1), Base64.Buffer(UInt8[0xf0, 0xee, 0x9c, 0x80, 0x57

[johnnychen94]

What should be the expected behavior here? clamp01 or linear contrast adjustment (e.g., (x .- min(x)) ./ (max(x)-min(x))?

[tlnagy]

What makes the most sense to me is to remap negatives as one color and positives as another say (green and magenta) to preserve the sign information. Neither clamping nor linear contrast adjustment does that. Why else would the user be using a signed FixedPointNumber if they didn't want the signed info preserved?

So have the type range:

julia> Gray(typemin(Q0f31))
Gray{Q0f31}(-1.0Q0f31)

julia> Gray(typemax(Q0f31))
Gray{Q0f31}(0.9999999995Q0f31)

map linearly to green and magenta, respectively. With zero equal to black.

[johnnychen94]

I'm not sure if this should be the default behavior because people may have different tastes. We might apply clamp01! as the default option (because it's the fastest and it's consistent with float numbers) and leave some helper functions for this purpose.

Would you mind uploading one or two images to play with?

[tlnagy]

That's fair, but then why would the user be using a signed FixedPointNumber if we're just going to throw away the negatives?

An easy example is

julia> rand(Gray{Q0f31}, 100, 100)

[johnnychen94]

Okay, I see your point. There's nothing special about FixedPointNumbers here; if we want to preserve the out-of-range value information, then we should treat it in a type-agnostic way.

This issue should be fixed in two steps:

1. support signed FPN properly still using clamp01
2. change the default mapi implementation to somehow display the out-of-range information.

The way you proposed for Gray sounds good to me, now I'm more concerned with RGB values.

---

There's nothing special about FixedPointNumbers here; if we want to preserve the out-of-range value information, then we should treat it in a type-agnostic way.

I'm saying that overly using the type information to change the behavior could sometimes be problematic. Just a quote from tim:

JuliaImages/ImageTransformations.jl#59 (comment)

Just to clarify a little bit more: where possible I tend to think of types as an implementation detail; using them to convey meaning should be done only when it increases clarity, and then you have to think carefully about what other methods are defined for those types.

[tlnagy]

I ran into this issue again. I think it's less important exactly how we display the image, we can pick one thing and stick with it.

The current behavior is still the following, which is quite annoying to deal with in an interactive environment like IJulia:

julia> using ImageShow, FixedPointNumbers, Colors                                                                                                                       
                                                                                                                                                             
julia> rand(Gray{Q0f31}, 100, 100)    
All errors:
===========================================
ArgumentError: Q0f31 is a 32-bit type representing 4294967296 values from -1.0 to 1.0; cannot represent 1
===========================================
ArgumentError: Q0f31 is a 32-bit type representing 4294967296 values from -1.0 to 1.0; cannot represent 1
===========================================
FileIO.MimeWriter writer error: neither save nor fileio_save is defined
  due to MethodError(FileIO.MimeWriter.var"#save##kw"(), ((mapi = ImageShow.var"#8#10"(),), FileIO.MimeWriter.save, FileIO.Stream{FileIO.DataFormat{:PNG}, IOContext{Base64.Base64EncodePipe}, Nothing}(IOContext(Base64.Base64EncodePipe(IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1), Base64.Buffer(UInt8[0xf0, 0xee, 0x9c, 0x80, 0x57

I'm fine with simply clamping to positive values for display as long as we throw a warning that we're modifying the data for display. That should work for both RGB and Gray values.