Click here to Skip to main content
15,867,308 members
Articles / Mobile Apps / Android

Catalano Framework

,
Rate me:
Please Sign up or sign in to vote.
4.98/5 (37 votes)
27 Aug 2014CPOL10 min read 116.3K   3.6K   50   37
Scientific computing for Java and Android.

Introduction

Hello all! My name is Diego Catalano, and I am a Java programmer. And this for a short while had been a problem, because I am a huge fan of the AForge.NET and Accord.NET Frameworks for image processing and scientific computing in .NET.  Unfortunately, this meant I couldn't use either framework in one of my favorite environments: Android.   

In this article, I am going to present an ongoing effort to bring the best features from both frameworks to Java; and, in turn, give a hint on what we can do once we get all this power into mobile computing. I am going to present the Catalano Framework, a framework for scientific computing written in Java with maximum portability in mind. Here I am getting some help from Accord.NET's author, César Souza, to not only bring the newest additions into my own framework, but to also contribute my own implementations and discoveries into Accord.NET.   

Also, let me get this straight. Due to my knowledge on using both AForge.NET and Accord.NET I have been rewarded with a job. My desire, today, is thus to return the favor and reward the community with my own contributions. A tool to bring scientific computing features to desktop, server and mobile applications,  all in a single shell.   

Introduction   

The main idea was to extend, improve and port most classes from both frameworks to Java. Thus said, I've established the same well known architectural division found on the aforementioned libraries, using distinct namespaces for distinct needs.  

If you are already familiar with any of the two projects, you should feel at home. The framework is divided into the following namespaces:  

  • Catalano.Image  
  • Catalano.Fuzzy 
  • Catalano.MachineLearning 
  • Catalano.Math 
  • Catalano.Neuro  
  • Catalano.Statistics  

In order to use the framework, just add library references as you would do in AForge.NET or Accord.NET. In NetBeans, you can add references to the .jar files in the same way you would load a .dll library in Visual Studio or SharpDevelop.  

Image 1

If you are using Eclipse to develop an Android application (i.e. using the Android SDK), there is only one slight difference you will need to do: Instead of referencing Catalano.Image.jar, add a reference to Catalano.Android.Image.jar instead. 

 Image 2

This is all you need to do to be up and running with the library. Now let's see what we will be able to use it for! 

Imaging Processing Library

Image processing is one of the core AForge.NET Features. Besides providing all implementations found on the aforementioned framework, I've added several new image processing algorithms, specially suited for a variety of applications. One of the applications I would like to show is the automatic binarization of image documents

Let the two images below demonstrate how we can recover a document using Bradley Local Thresholding:

Original Image:                                                            Processed Image:

 Image 3                    Image 4

 

Here is the code I've used to make this work: 

Java
//Load an image
FastBitmap fb = new FastBitmap("bradley.png");

// Convert to grayscale
Grayscale g = new Grayscale();
g.applyInPlace(fb);

// Apply Bradley local threshold
BradleyLocalThreshold bradley = new BradleyLocalThreshold();
bradley.applyInPlace(fb);

//Show the result
JOptionPane.showMessageDialog(null, fb.toIcon(), "Result", JOptionPane.PLAIN_MESSAGE);

As it might be seen, here we are using the almost exact interfaces given by the AForge.NET Framework. This should make it easy to port knowledge from a framework to the other with minimum work. Besides the Bradley Local Threshold, in the Catalano Framework you will find several other algorithms, like 

  1. Bernsen Threshold
  2. Niblack Threshold
  3. Sauvola Threshold
  4. Wolf Jolion Threshold   

If you are not into image binarization, let's find another example: Let's find some interest points in a scene. 

Image 5

Corner detection algorithms have several uses, including some uses already shown here in other CodeProject articles. Among other things, they can be used to match points between images, to perform feature extraction for posterior image classification, scene understanding, image registration, among several others.  

In the Catalano Framework, corners detection follows almost exactly the interfaces found in Accord and AForge.NET. The example below shows how to apply Susan's corners detection to identify interest points in a sample image: 

Java
//Load an image
FastBitmap fb = new FastBitmap("blocksTest.gif");

Grayscale g = new Grayscale(Grayscale.Algorithm.Average);
g.applyInPlace(fb);

SusanCornersDetector susan = new SusanCornersDetector();
ArrayList<IntPoint> lst = susan.ProcessImage(fb);

//If you want to detect using Harris
//HarrisCornersDetector harris = new HarrisCornersDetector();
//ArrayList<IntPoint> lst = harris .ProcessImage(fb);

fb.toRGB();
for (IntPoint p : lst) {
    fb.setRGB(p, 255, 0, 0);
} 

Susan Corners Detector                              Harris Corners Detector

Image 6 Image 7

Imaging Processing Library (Android)

Now, here comes the real power of this framework. Let's see how we can achieve the same things we've done before, but this time from within an Android device. The following code reproduces the first example given in this article, binarizing a sample image using Bradley's Local Thresholding algorithm, but directly from an Android device

Java
ImageView image = (ImageView)findViewById(R.id.imageView1);

//Load an image
Bitmap bm = BitmapFactory.decodeResource(getResources(), R.drawable.sample20);

//Load Fast bitmap
FastBitmap fb = new FastBitmap(bm);

//Convert to grayscale
Grayscale g = new Grayscale();
g.applyInPlace(fb);

//Apply Bradley local threshold
BradleyLocalThreshold bradley = new BradleyLocalThreshold();
bradley.applyInPlace(fb);

//Display the result
image.setImageBitmap(fb.toBitmap()); 

Image 8

Several other image processing algorithms are also available under this same shell. Other examples include edge detection algorithms, such as the Sobel operator. Being an edge detection algorithm means this algorithm can highlight any borders or boundaries presented in an image. Its usage is completely similar in both Android and standard Java applications.  

The following example demonstrates how to run the Sobel Edge Detector in Android. 

Java
//Convert to grayscale
Grayscale g = new Grayscale();
g.applyInPlace(fb);

//Apply Bradley local threshold
SobelEdgeDetector sobel = new SobelEdgeDetector ();
sobel.applyInPlace(fb);

Image 9  

I hope this can get most of you interested - but note we have just started scratching the surface of what is available in the framework! 

Improvements 

One of the oldest features available on AForge.NET is the ability to perform a Fast Fourier Transform (FFT) in an image. This feature transposes an image from the space domain to the frequency domain, allowing the use of several interesting image processing filters such as frequency-pass filters or even helping improving convolution through the Convolution-in-space/Multiplication-in-frequency theorem. 

However, there is a catch present on most FFT implementations, including the one available in AForge.NET. The problem is that the FFT, by definition, can only be performed on images which have both width and height as powers of two. This means it is fine to compute a FFT for 64x64, 512,512, 1024x1024 images, but if we attempt to compute the transform on a 592x634 image, it will not work. 

Please note that this is not a limitation of the implementations per se,  but rather a consequence of one of the basic assumptions that make the Fast Fourier Transform, well, Fast in the first place. In order to circumvent this, one can use a feature known as zero-padding. 

Zero-padding simply adds extra zero-valued pixels to an image until its size forms a power of two. After that, we compute the FFT as normal, and then we remove the extra elements we added in the first step. As such, please consider the following example using AForge.NET in C#: 

C#
Bitmap image = new Bitmap("test.bmp");
image = new GrayscaleBT709().Apply(image);
 
ComplexImage ci = ComplexImage.FromBitmap(image);
ci.ForwardFourierTransform(); 

Here, if test.bmp is not test.bmp does not have a size which is a power of two, we will likely encounter an AForge.Imaging.InvalidImagePropertiesException.  

Now, in the following example, we will not only perform a Fourier Transform, but also run a frequency filter to extract low frequencies from an arbitrarily sized image: 

Java
FastBitmap fb = new FastBitmap("c:\\files\\test.bmp");
fb.toGrayscale();
JOptionPane.showMessageDialog(null, fb.toIcon(), "Image", JOptionPane.PLAIN_MESSAGE); 

FourierTransform ft = new FourierTransform(fb);
ft.Forward();
fb = ft.toFastBitmap();
JOptionPane.showMessageDialog(null, fb.toIcon(), "Fourier Transform", JOptionPane.PLAIN_MESSAGE);
        
FrequencyFilter ff = new FrequencyFilter(0, 60);
ff.ApplyInPlace(ft);
fb = ft.toFastBitmap();
JOptionPane.showMessageDialog(null, fb.toIcon(), "Frequency Filter", JOptionPane.PLAIN_MESSAGE);
        
ft.Backward();
fb = ft.toFastBitmap();
JOptionPane.showMessageDialog(null, fb.toIcon(), "Result", JOptionPane.PLAIN_MESSAGE);

The results are shown below. We can see how the non-square image was transformed into the frequency domain, filtered, and then converted back to the space domain in the same, previous original size.   

 

 

 

Original Image (480x361)

 

Image 10 
Fourier Transform (512,512)
Image 11 
   
Frequency Filter (512,512) 
Image 12

Result image (480,361)

 

Image 13

Benchmark   

I've decided to dedicate a section to present a benchmark between the Catalano framework and the standard features available in AForge.NET and Accord.NET. Here, the goal is not to prove one is better than the other, but instead is to provide some baseline comparisons to indicate what can be expected from the current implementations.   

All tests have been repeated 100 times. Initial executions (before those 100 times) have been discarded to let any possible compiler/JIT optimizations quick in before the actual measurements.  All tests have been made outside Visual Studio, i.e. by going to the binary folders and running the executables directly. All binaries have been compiled in Release mode, for Any CPU (meaning they were ran in 64-bits mode). All tests have been ran using the same machine with the following specifications: 

Processor: Intel® i7-2670QM CPU @ 2.20Hz, 4 cores (8 in HyperThread), 6 MB cache
RAM: 8 GB, DDR3 1600MHz   

The first test started with a simple Median filter. In this test, we processed a 1024 x 770  RGB image using a 5x5 kernel window.   

Image 14

In this particular test, the difference between the Catalano Framework (non concurrent) and AForge.NET is roughly 10s. If we consider a parallel implementation, which should make use of all 4 cores in the test machine, the difference grows even more, to a whooping 71 seconds. 

The second test started with a simple Sobel Edge Detector. In this test, we processed a 1024 x 770  Grayscale image using both parallel, direct, and AForge.NET's default implementation.   

 

Image 15

As we can see here, the difference between the Catalano Framework (non concurrent) and AForge.NET is of just 467 ms. The concurrent version improves further on this mark, being roughly twice as fast as the original AForge.NET implementation. As the test machine had 8 virtual cores, this means the algorithm is not straightly easily parallelizable.   

Now, on with the third test. This test involved measuring the time it takes to process a Gaussian Blur over a 1024 x 770  RGB image. Results are shown below.

Image 16
 

In this particular test, we can see that AForge.NET outperforms Catalano with a margin of at least 4 seconds. Again we can see both implementations are comparable, while AForge.NET scores a little better at this test.  

The fourth test was done against a Harris Corners Detector from Accord.NET. In this test, we had processed a 1024 x 770  Grayscale image, with results shown below. 

 Image 17

As we can see, Accord.NET scores slightly better at this test, but the difference is very comparable and equivalent among the two implementations. This is an indication both C# and Java can also shown very similar performance on such applications.  

Finally, the last test was done by comparing the FAST Corners Detector in in both Catalano and Accord.NET. This test involved the processing of a 1024 x 770  Grayscale image using FAST-9, which is a particular version of the FAST detection. Results are shown below.   

 Image 18

As it can be seen, this time the difference between Catalano Framework and Accord.NET was actually very noticeable, being around 23649 ms. This test clearly reveals that Accord.NET's implementation outperforms the Catalano Framework implementation. As we can see, there is much room for improvement and work to be done - but mostly, the overall intent, which was to bring image processing into Android, has mostly been achieved! 

Conclusion   

In this article, I've attempted to present the Catalano framework to the world. Here we have seen how this relatively new framework complements the open-source ecosystem by offering a new way to create image processing and scientific computing applications in Java by unifying the best features from the Accord and AForge.NET Frameworks. Development is on full steam and is being ran at Github.  

During the time I've started building the framework until now,  I acquired a lot of knowledge in several different areas, starting from the scientific method up to software engineering and systems design. For that, I am quite grateful for all the good people who I found during the way, including the fellow researches who agreed to share their code under LGPL-compatible licenses so they could be integrated in the framework.   

I hope you will enjoy the framework the same amount I've enjoyed working on it!  

Future works 

From now, there are several areas I intend to improve. Almost all of my contributions are being gradually shared with César (from the Accord.NET Framework) to be integrated into his framework; and other features are also being ported from .NET to Java with some help. 

 

I also intend to create a blog explaining each filter in Catalano Framework, and present some actual sample applications demonstrating its features. There is also a plan to move the repository to GitHub to make code contributions easier. 

There are also some key points I would like to continue working on:   

  • Port to Google Glass® (I would gladly accept one for testing purposes :-P ).
  • Convert all filters to use multi-threaded, parallel processing. 
  • Add a library for Biometrics related algorithms. 
  • Continue working, extending and improving the whole framework. 

History 

  • 01 November 2013 - First version published. 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Engineer NAVER LABS Europe
France France
Computer and technology enthusiast, interested in artificial intelligence and image processing. Has a Master's degree on Computer Science specialized on Image and Signal Processing, with expertise on Machine Learning, Computer Vision, Pattern Recognition and Data Mining systems. Author of the Accord.NET Framework for developing scientific computing applications.

If you would like to hire good developers to build your dream application, please check out DaitanGroup, one of the top outsourcing companies in Brazil. This company, located in Brazil's Sillicon Valley but with US-based offices, has huge experience developing telecommunications software for large and small companies worldwide.

Written By
Student
Brazil Brazil
I'm very interested in image processing and machine learning. In my free time I like to work out, run and to cook.

Comments and Discussions

 
QuestionClassification Pin
Member 1320399916-May-17 12:13
Member 1320399916-May-17 12:13 
Questionvisual studio Xamarin ? Pin
Ayad Kara Kâhya27-Apr-16 7:18
Ayad Kara Kâhya27-Apr-16 7:18 
QuestionA great thanks Pin
hzawary20-Aug-15 22:21
hzawary20-Aug-15 22:21 
QuestionUsing the framework with maven Pin
Olexij Tkatchenko5-Mar-15 22:41
Olexij Tkatchenko5-Mar-15 22:41 
Questionerror in fb.toBitmap (); Pin
Yogesh Chaudhary3-Feb-15 21:13
Yogesh Chaudhary3-Feb-15 21:13 
AnswerRe: error in fb.toBitmap (); Pin
Diego Catalano9-Feb-15 8:03
Diego Catalano9-Feb-15 8:03 
GeneralMy vote of 5 Pin
xfzheng1-Sep-14 15:07
xfzheng1-Sep-14 15:07 
QuestionImage Comparison using Catalano framework Pin
Member 1104482929-Aug-14 21:41
Member 1104482929-Aug-14 21:41 
AnswerRe: Image Comparison using Catalano framework Pin
Diego Catalano30-Aug-14 6:00
Diego Catalano30-Aug-14 6:00 
GeneralMy vote of 5 Pin
Volynsky Alex27-Aug-14 13:14
professionalVolynsky Alex27-Aug-14 13:14 
QuestionFastRetinalKeypoint Pin
xander2725-May-14 19:49
xander2725-May-14 19:49 
AnswerRe: FastRetinalKeypoint Pin
Diego Catalano19-Sep-14 4:33
Diego Catalano19-Sep-14 4:33 
QuestionAndroid memory problems Pin
Alan Żur23-Apr-14 22:44
Alan Żur23-Apr-14 22:44 
AnswerRe: Android memory problems Pin
Diego Catalano30-Aug-14 13:44
Diego Catalano30-Aug-14 13:44 
AnswerRe: Android memory problems Pin
Diego Catalano1-Oct-14 20:00
Diego Catalano1-Oct-14 20:00 
GeneralMy vote of 5 Pin
jpfolador11-Mar-14 13:43
jpfolador11-Mar-14 13:43 
QuestionHow to draw a Histogram by using your package "Catalano.Statistics" Pin
Malik Awais14-Feb-14 6:33
Malik Awais14-Feb-14 6:33 
AnswerRe: How to draw a Histogram by using your package "Catalano.Statistics" Pin
Diego Catalano8-Mar-14 8:26
Diego Catalano8-Mar-14 8:26 
Questionbitmap to pencil sketch Pin
Member 105677434-Feb-14 1:16
Member 105677434-Feb-14 1:16 
AnswerRe: bitmap to pencil sketch Pin
Diego Catalano4-Feb-14 1:31
Diego Catalano4-Feb-14 1:31 
GeneralRe: bitmap to pencil sketch Pin
Member 105677434-Feb-14 19:58
Member 105677434-Feb-14 19:58 
GeneralRe: bitmap to pencil sketch Pin
Diego Catalano8-Mar-14 8:19
Diego Catalano8-Mar-14 8:19 
Questionget Blob information Pin
omidlolo27-Dec-13 22:25
omidlolo27-Dec-13 22:25 
AnswerRe: get Blob information Pin
Diego Catalano4-Feb-14 5:20
Diego Catalano4-Feb-14 5:20 
GeneralRe: get Blob information Pin
omidlolo4-Feb-14 8:54
omidlolo4-Feb-14 8:54 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.