In content-based image retrieval, relevant feedback is studied extensively to narrowthe gap between low-level image feature and high-level semantic concept. In gen-eral, relevance feedback aims to improve the retrieval performance by learning withuser’s judgements on the retrieval results. Despite widespread interest, but feedback related technologies are often faced with a few limitations. One of the most obviouslimitations is often requiring the user to repeat a number of steps before obtaining theimproved search results. This makes the process inefﬁcient and tedious search for theonline applications. In this paper, a effective feedback related scheme for content-based image retrieval is proposed. First, a decision boundary is learned via SupportVector Machine to ﬁlter the images in the database. Then, a ranking function for se-lecting the most informative samples will be calculated by deﬁning a novel criterionthat considers both the scores of Support Vector Machine function and similarity met-ric between the ”ideal query” and the images in the database.