Click here to Skip to main content
15,886,788 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
There is a color reduction library called ExoQuant written by Dennis Ranke from MIT. There are versions available in JavaScript,C#,VB and Python. I have been experimenting with every color reduction library I can find. Different algorithms are better on different types of images. When I am done I hope to write a review with examples because there is a serious lack of information on using all of these libraries. They are free though. So many thanks to all who help out "challenged" developers like myself.

Anyway the ExoQuant library is based on Xiaolin Wu's Color Quantizer algorithm. The algorithm works best on images that have been giving me certain issues. I'd love to upload the images but I don't see a way to ddo that here. If someone reaches out I'd be happy to share them. They are not proprietary, they just give quantizers quite a bit of agita. Even if the color quant is 48 or 64.

the link to the libraries is https://github.com/DavidSM64/ExoQuantVB. There are other links to c#, javascript and python there too.

Here is the issue as best as I can describe it. When hashing each pixel which is stored in a byte() stream there is an overflow on any color passed to it on the third line of the subroutine:

Private Function MakeHash(ByVal rgba As Uinteger) As UInteger
            Try
                '1
                rgba -= (rgba >> 13) Or (rgba << 19)
                '2
                rgba -= (rgba >> 13) Or (rgba << 19)
                '3
                rgba -= (rgba >> 13) Or (rgba << 19)
                rgba -= (rgba >> 13) Or (rgba << 19)
                rgba -= (rgba >> 13) Or (rgba << 19)
                rgba = rgba And (EXQ_HASH_SIZE - 1)
                Return rgba
            Catch ex As Exception
                MessageBox.Show("ExoQuant.MakeHash: " + ex.Message + " " + ex.StackTrace)
                Return 255
            End Try
        End Function


The function is called from:

Public Sub Feed(ByVal pData As Byte())
            Dim channelMask As Byte = &HFF00 >> pExq.numBitsPerChannel
            Dim nPixels As Integer = Int(pData.Length / 4)

            For i As Integer = 0 To nPixels - 1
                Dim r As Byte = pData(i * 4 + 0),
                    g As Byte = pData(i * 4 + 1),
                    b As Byte = pData(i * 4 + 2),
                    a As Byte = pData(i * 4 + 3)
                'If pdata(i * 4 + 3) = 0 Then
                '    a = 255
                'Else
                '    a = pData(i * 4 + 3)
                'End If
                Dim hash As UInteger = MakeHash(ToRGBA(r, g, b, a))
                Dim pCur As ExqHistogramEntry = pExq.pHash(hash)

                While pCur IsNot Nothing AndAlso (pCur.ored <> r OrElse pCur.ogreen <> g OrElse pCur.oblue <> b OrElse pCur.oalpha <> a)
                    pCur = pCur.pNextInHash
                End While

                If pCur IsNot Nothing Then
                    pCur.num += 1
                Else
                    pCur = New ExqHistogramEntry()
                    pCur.pNextInHash = pExq.pHash(hash)
                    pExq.pHash(hash) = pCur
                    pCur.ored = r
                    pCur.ogreen = g
                    pCur.oblue = b
                    pCur.oalpha = a
                    r = r And channelMask
                    g = g And channelMask
                    b = b And channelMask
                    pCur.color.r = r / 255.0F * SCALE_R
                    pCur.color.g = g / 255.0F * SCALE_G
                    pCur.color.b = b / 255.0F * SCALE_B
                    pCur.color.a = a / 255.0F * SCALE_A

                    If pExq.transparency Then
                        pCur.color.r *= pCur.color.a
                        pCur.color.g *= pCur.color.a
                        pCur.color.b *= pCur.color.a
                    End If

                    pCur.num = 1
                    pCur.palIndex = -1
                    pCur.ditherScale.r = -1
                    pCur.ditherScale.g = -1
                    pCur.ditherScale.b = -1
                    pCur.ditherScale.a = -1
                    pCur.ditherIndex(0) = -1
                    pCur.ditherIndex(1) = -1
                    pCur.ditherIndex(2) = -1
                    pCur.ditherIndex(3) = -1
                End If
            Next
        End Sub


The ToRGBA function is:
Private Function ToRGBA(ByVal r As UInteger, ByVal g As UInteger, ByVal b As UInteger, ByVal a As UInteger) As UInteger
         Return r Or (g << 8) Or (b << 16) Or (a << 24)
     End Function


What I have tried:

The error message from line 3 is:

Arithmetic operation resulted in an overflow.

I have tried sending various bytestreams of color in both 24 and 32 bit images. I have set the alpha channel to 0,128 and 256. I've tried playing around with the bit shifting but I can't say I really understand what the recursion of the bit shift in the following does. There is also a lack of documentation and examples for logical it shifting in vb .net. I found the c# pretty confuding too.

Any help would be appreciateded.

The gitHub Links

rgba -= (rgba >> 13) Or (rgba << 19)
     rgba -= (rgba >> 13) Or (rgba << 19)
     rgba -= (rgba >> 13) Or (rgba << 19) ' blows up here value of rgba before this line is 61151760
     rgba -= (rgba >> 13) Or (rgba << 19)
     rgba -= (rgba >> 13) Or (rgba << 19)
     rgba = rgba And (EXQ_HASH_SIZE - 1)
Posted
Updated 4-Feb-20 8:31am

You'll probably have better luck by asking the library maintainer(s) than us. Try opening an Issue on their GitHub repository, at Issues · DavidSM64/ExoQuantVB · GitHub[^].
 
Share this answer
 
Comments
larry118 4-Jan-20 17:05pm    
I thought the post was closed. However I went back and was able to post. However I always get the best help here. Mind if I leave it up a bit?
Thomas Daniels 4-Jan-20 17:08pm    
I don't find it a problem at all that you leave this post up here :) I only think it's way less likely that we can help you out than the maintainers.
larry118 4-Feb-20 14:34pm    
Someday I'll learn how to type.
Hi.

Simplest solution is to turn off overflow in compile options. Also the big hug code for th has been updated so you won't even have to do that. Thanks to DavidSM64 for fixing the code I github. The color reduction is the best I've seen.

Larry
 
Share this answer
 
Comments
phil.o 4-Feb-20 14:51pm    
Thanks for the update. You may mark the question as answered, so that it leaves the queue :)

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900