Friday, October 30, 2009

Table of results for MNIST

This is a table documenting some of the best results some paper obtained in MNIST dataset.

Results shown indicates the error obtained by training on all 60,000 samples and testing on 10,000 samples.
  1. Efficient Learning of Sparse Representations with an Energy-Based Model (2006)
    0.39%
  2. Unsupervised learning of invariant feature hierarchies with applications to object recognition (2007)
    0.64%
  3. Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations (2009)
    0.82%
  4. Deep Boltzmann Machines (2009)
    0.95%
  5. CS81: Learning words with Deep Belief Networks (2008)
    1.12%
  6. Reducing the dimensionality of data with neural networks (2006)
    1.2%
  7. Deep learning via semi-supervised embedding (2008)
    1.5%

Sunday, October 25, 2009

Table of results for Caltech 256

This is a table documenting some of the best results some paper obtained in Caltech-256 dataset.
  1. Image Classification using Random Forests and Ferns (2007)
    Cited 51 times. 45.3%

Friday, August 21, 2009

Table of results for Caltech 101

This is a table documenting some of the best results some paper obtained in Caltech-101 dataset.

Results shown here are all trained using 30 samples from each category.
  1. Image Classification using Random Forests and Ferns (2007)
    Cited 51 times. 81.3%
    Additional Info: Bosch Multi-way SVM
  2. In Defense of Nearest-Neighbor Based Image Classification (2008)
    Cited 22 times. 79.23%
    Additional Info: NBNN (5 descriptors)
  3. Visual Geometric Group (VGG)'s implementation of Multiple Kernel Image Classifier trained on dense SIFT, self-similarity, and geometric blur features
    78.59% (Without taking into account the BACKROUND_GOOGLE class)
  4. Representing shape with a spatial pyramid kernel (2007)
    Cited 70 times. 77.8%
    Additional Info: Result of 77.8% is obtained by combining all 4 cues (shape 180, shape 360, gray appearance and color appearance.
  5. Linear Spatial Pyramid Matching Using Sparse Coding for Image Classification (2009)
    Cited 1 time. 73.2%
    Additional Info: ScSPM (Linear Spatial Pyramid Matching - uses linear kernel on spatial-pyramid pooling of SIFT sparse codes)
  6. Recognition using Regions (2009)
    73.1%
  7. Learning Coupled Conditional Random Field for Image Decomposition with Application on Object Categorization (2008)
    70.38%
  8. Fast Image Search for Learned Metrics (2008)
    Cited 14 times. 69.6%

    Additional info: ML+CORR
  9. Caltech-256 Object Category Dataset (2007)
    Cited by 99. 67.6%
    Additional Info: Griffin's SPM
  10. SVM-KNN - Discriminative Nearest Neighbor Classification for Visual Category Recognition (2006)
    Cited 164 times. 66.23%
  11. Image Retrieval and Classification using Local Distance Functions (2006)
    Cited 42 times. 66%
  12. Bag-of-Features Kernel Eigen Spaces for Classification (2008)
    65.5%
  13. Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations (2009)
    Cited 4 times. 65.4%
  14. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories (2006)
    Cited 418 times. 64.6%
  15. Kernel Codebooks for Scene Categorization (2008)
    Cited 11 times. 64.12%
  16. Visual Word Ambiguity (2010)
    64.1%
  17. Using dependent regions for object categorization in a generative framework (2006)
    Cited 54 times. 63%
  18. SIFTing the Relevant from the Irrelevant - Automatically Detecting Objects in Training Images (2009)
    61.45%
  19. Pyramid Match Kernels: Discriminative Classification with Sets of Image Features (2006)
    Cited 276 times. 58.2%
  20. Max-Margin Additive Classifiers for Detection (2009)
    56.49%
  21. Multiclass Object Recognition with Sparse, Localized Features (2006)
    Cited 143 times. 56%
  22. Efficiently Matching Sets of Features with Random Histograms (2008)
    Cited 2 times. 54.1%
  23. Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition (2007)
    Cited 39 times. 54%
  24. Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition (2008)
    Cited 7 times.
    53%
  25. Exploiting Unlabelled Data for Hybrid Object Classification (2005)
    Cited 10 times. 43%
  26. Object Recognition with Features Inspired by Visual Cortex (2006)
    Cited 191 times.
    42%
  27. Exploiting Unlabelled Data for Hybrid Object Classification (2005)
    Cited 10 times. 17%

Sunday, April 05, 2009

Re-focused

In an attempt to re-focuses my energy on IT-related stuff that I will be doing, I have been doing some thinking and came up with a list (The list is not final):

Research
(I did not say machine learning, nor pattern recognition, as those are just means to an end)
- Computer Vision
- Natural Language Processing

Development
(I did not say functional programming, as those are just means to an end)
- Writing parallel & concurrent programs (Parallel processing, Concurrent programming)
- Writing maintainable, beautiful code

Design
- Designing simple, practical & nice UI

Friday, March 06, 2009

Return to research

This is a post foreshadowing my come back to the research world. I've been thinking a lot about vision. How do we humans perceive things? How do we recognize things? How do we get a sense of deja vu after seeing things we have seen before?

Even though I do not have the answer to these perplexing questions. I do however think that I *might* have a solution that might solve it to a certain degree. That's a pretty modest statement. But I'm not going to claim anything more exotic before I come up with a working prototype.

The ideas I have in mind have a lot to do with time-based learning, that is, learning not just things but the relationships they have. That is, that certain meaningful things appear to have an order. For example, you do not see a cat walking by, then suddenly it become a dog for half a second, then magically becoming a cat again. Things are not random. They appear in logical order. The idea is nothing new. It has been addressed by Jeff Hawkins in his work on Hierarchical Temporal Memory.

End of part 1. I will talk more about this later.

Monday, December 29, 2008

Programming skill...

Programming skill...
  • Those that has it, talks about why there are people who just can't program (no matter what).
  • Those that don't have it, talks about why the world isn't fair.
Why do I say this? Because if I see another article/blog that says some people just can't program no matter what, I need to pull his head off.

Disclaimer: In no way did I claim I'm superior in programming skills.

Sunday, June 08, 2008

Why good is actually bad.

Basic notion has it that as we gain more experience, our peers or children gets to reap the benefits. Well, the problem is, that is where the problem lies. You see, human learns by experience, be it good or bad. When we made a mistake, some external signals, be it your teacher or your mum tells you not to do it again (Bad experience). When we did something right, you are rewarded or praised (Good experience). Call it reinforcement learning, call it supervised learning, the point is, if you did something wrong, if the signs that you did something wrong was hidden from you, you are learning to repeat your wrongdoing. You may not implied that, or want to, but believe it or not, you are (asking him to repeat the wrong doing). There is no such thing as, "You are doing it wrong, but I'll just let you go this time.". It makes absolutely no sense. If you look at this way, this is common sense. We may not like being scold or make mistakes, it's all aweful medicine, but sometimes the patient just needed it.

The scary thing is, that’s what we do every now and then. Let me tell of a fictional story to illustrate the problem. A software programmer discovered that with C++, it is difficult to write code that does not leak memory, he suggested the team to use Python instead, before the team has decide which language to use. All is good and well. Well, the problem is C++ is still prevalent. Many legacy systems are still written in C/C++. When one of the other programmers in the team was assigned to maintain legacy software written in C++, he might not perform. Why? He/She has never learned best practices of C++! And the sole reason of that is because he/she never gets to learn it anyway; every time the chances come one of the programmer in the team will recommend the team to use a more modern, safer language. So you see, the point is, if you are only told that it's a wrong way and you should not go there, you think that it has saves you whole lot of trouble, but the truly sad truth is, it does not mean that you have actually learn anything. You merely avoided the problem.

Now, some of you might argue that there is simply too much to learn, it is unfeasible to learn everything. While that is true, if that something to learn is something fundamental, it is a mistake not to go through the entire learning process. This is the case where shortcut is not a good idea. Like the famous saying goes, “If we do not learn from history, we are bound to repeat it”. And the only way to learn history is to be in history. Walk in their shoes. Doing what they are doing. Merely being told the mistakes of the past does nothing to prevent it from happening it again, since there may be new, unexplored alternative path that will ultimately lead to the same mistake. And the only way to prevent it from happening is to have greater understanding of the nature of the mistake, and learning how to prevent it from ever happening again.

Perhaps, this is why there has been talk that the quality of Computer Science graduates are dropping, due to the fact that Java was chosen as most CS first language, where as lower level languages like assembly and C/C++ should be taught first, because they not only taught the students about the language itself, but also the fundamentals of computer architecture itself, how things work in the low level and etc. You can read all about it in [Computer Science Education: Where Are the Software Engineers of Tomorrow?].

The truth is, it is extremely dangerous to say that all history is useless experience, though that is not to say that there is no such thing as useless experiences. If you studied machine learning, you will know that, a typical artificial neural network would become brain dead if it is only showed positive samples without any negative samples during the training phase. A human brain works almost in the same principle, since a human brain is really both a discriminative and a generative system. Throwing off the discriminative part of our brain and it is likely that we will never learn. Similarly, if a baby is only taught the right way of doing things, the baby will not have a clue of if what he is doing is right or wrong. In other words, his discriminative ability, his ability to discern what is right or wrong, what is good or bad is simply not there or severely random and thus flawed.

Even if there are times when we needed to choose which knowledge to gain, I shudder to think at the outcome of making the wrong judgement on which knowledge is useless, and which is not: Because depending on what the situation is, it's actually very hard for us to really know for sure. Like Steve Jobs said, it's impossible for us to connect to dots ahead of time. We can only do so, looking back.

This is all obvious and understandable. However, recently, it is increasingly becoming harder to made mistakes, as we are becoming more intolerant to mistakes, while technology is becoming more adept at hiding complexities that are deemed to cause human mistakes. In the office, you are not allowed to use tools that your project leader deemed is bad, you will be fired for making the wrong decisions; In the university, you can't opt for subjects that are deemed to be obsolete by some, you are forced to answer questions based on what is stated on the textbook instead of your own thinking, which ultimately affected our creativity as mentioned in this article; In the house, children are taught not to talk to strangers, just like what Bruce Scheneier in his article "The Kindness of Strangers". In almost every cases, one is simply not acceptable to be different from others, you have to be just like everyone else, even though everyone else could be wrong. We should be given guidance, yes, but more importantly, we should be given the chance to make mistakes! Not repeating the same mistakes should be everyone's responsibility. Not the enforcer. Ideally.

To summarize, discriminative capability is important for us to learn. It is extremely crucial that we are left to made mistakes so that we would not do it again. The experience of doing something wrong is much more important than the experience of doing something correctly. Now, if there is an alternative way for us to experience the whole crucial process of making mistakes without actually wasting any time for it and yet learn a good lesson out of it, I would really like to know about it.

Monday, May 05, 2008

End of examination. Begining of research life.

Today marks the last day of my examination in this 3 year UTAR life. And I feel like this is the worst examination of all, and the only paper that I have the confidence to get at least A- is AI. To me, doing good in examination just don't motivate me that much anymore, what motivates me more now is research and development in things related to applied AI and computer vision. Which although sounds hard, is actually a lot easier than you might think, because there is no fixed and established way of doing things. Therefore, it tends to tax your creativity more. It's a lot harder to study, understand and memorizes things that other genius scientist/engineer has established, which is what is expected of you in university. In world of research, you don't necessary have to do it using other scientists' ways. They could be wrong and there seems to be always room for improvement. Everyone is solving a small piece of the puzzle, and no one knows the complete solution.

However, before I officially begin my research life, I will take a detour to work in Panasonic R&D for 1 year or so, commencing on June, before coming back to UTAR to begin my Master degree course.

Friday, February 01, 2008

My AI mini project

UCEC3064 - Intelligent Technique for Engineering Applications

Proposed mini project title:
Improving Circular Pairwise Neural Network Performance on Multiclass Classification Problems

Abstract:
Traditionally, neural networks are trained in a monolithic fashion. That is, a neural network would be trained to classify all K classes, preferably given equivalent amount of training samples for each classes. Training is slow and is unable to take advantage of current multi-cores or multi-processor (SMP) systems to train different classes concurrently. The research done by Teo Choon Hui on circular pairwise classification is an attempt to remedy this by breaking down a K class problem into k binary circular pairwise classification sub-problems. However, this method will reduce recognition accuracy due to the fact that there is a lacked of direct competition between certain pairs of classes. But, at the same time, such a method reduces training time by almost a factor of 3. In this research, we will compare the results of a binary one-versus-all classifier and a pairwise classifier with a circular pairwise classifier as proposed by Teo Choon Hui and attempts to improve recognition accuracy by experimenting with selective circular pairwise classification, in which hard binary sub-problems are paired instead of randomly choosing any 2 classes. We will then attempts to introduce an N-th class into each binary classifier to become a ternary circular pairwise classifier and evaluate its performance.

Keywords:
One-versus-all, Binary classification, Pairwise, Single-winner election methods, Circular Pairwise